IBM 88743BU User Manual
IBM 88743BU - System x3950 E Manual
 |
UPC - 000435945945
View all IBM 88743BU manuals
Add to My Manuals
Save this manual to your list of manuals |
IBM 88743BU manual content summary:
- IBM 88743BU | User Manual - Page 1
Front cover Planning, Installing, and Managing the IBM System x3950 M2 Understand the IBM System x3950 M2 and IBM x3850 M2 Learn the technical details of these high-performance servers See how to configure, install, manage multinode complexes ibm.com/redbooks David Watts Jens Reizel Paul Tan Kevin - IBM 88743BU | User Manual - Page 2
- IBM 88743BU | User Manual - Page 3
International Technical Support Organization Planning, Installing, and Managing the IBM System x3950 M2 November 2008 SG24-7630-00 - IBM 88743BU | User Manual - Page 4
information in "Notices" on page ix. First Edition (November 2008) This edition applies to the following systems: IBM System x3950 M2, machine types 7141 and 7233 IBM System x3850 M2, machine types 7141 and 7233 © Copyright International Business Machines Corporation 2008. All rights reserved. Note - IBM 88743BU | User Manual - Page 5
Scalable upgrade option for x3850 M2 11 1.3 Multinode capabilities 14 1.4 x3950 M2 Windows Datacenter models 15 1.4.1 IBM Datacenter Unlimited Virtualization offering 16 1.4.2 IBM Datacenter Unlimited Virtualization with High Availability . . . . . 16 1.4.3 Upgrading to Datacenter Edition 17 - IBM 88743BU | User Manual - Page 6
path diagnostics 47 1.12.2 BMC service processor 48 1.12.3 Remote IBM BladeCenter and iDataPlex 57 2.3 Comparing x3850 M2 to x3850 59 2.4 Comparing x3950 M2 to x3950 62 2.5 System scalability 64 2.6 Operating system scalability 66 2.6.1 Scaling VMware ESX 68 2.6.2 Scaling Microsoft Windows - IBM 88743BU | User Manual - Page 7
PCI device scan order 188 3.5.3 PCI adapter installation order 189 3.5.4 PCI Express device-related information in the BIOS 190 3.5.5 Supported PCI Express adapter options 194 Chapter 4. Multinode hardware configurations 195 4.1 Introduction and terminology 196 4.2 Multinode capabilities 198 - IBM 88743BU | User Manual - Page 8
Downloading the firmware 247 5.1.3 Performing the updates 247 5.2 Confirming BIOS settings 254 5.3 Supported operating systems 257 5.3.1 VMware ESX operating systems 259 5.3.2 Windows BMC firmware update 307 6.1.8 Installing the BMC device drivers 308 6.1.9 Ports used by the BMC 316 6.2 - IBM 88743BU | User Manual - Page 9
6.2.8 MIB files 333 6.2.9 Error logs 333 6.3 Use of IBM Director with IBM Director: Implementation of servers 346 6.5.1 Integrating x3850 M2 and x3950 M2 into IBM Director 347 6.5.2 Level 0: Implementation by service processors 348 6.5.3 Level 1: Implementation by the IBM Director Core Services - IBM 88743BU | User Manual - Page 10
viii Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 11
to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM - IBM 88743BU | User Manual - Page 12
or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also both. BitLocker, Hyper-V, Microsoft, SQL Server, Windows NT, Windows Server, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United - IBM 88743BU | User Manual - Page 13
UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Notices xi - IBM 88743BU | User Manual - Page 14
xii Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 15
for extremely complex, compute-intensive applications that require four sockets, plus processing power and large memory support. The x3950 M2 connected to form a single-system image comprising of up to 16 six-core processors, up to 1 TB of high speed memory, and support for up to 28 PCI Express - IBM 88743BU | User Manual - Page 16
systems, management hardware, and Windows®, Linux®, and VMware® operating systems. Paul Tan works as a presales System x, BladeCenter and Storage Technical Specialist at IBM Systems and Technology Group in Melbourne, Australia. He regularly leads customer presentations and solution workshops based - IBM 88743BU | User Manual - Page 17
Chia-Yu Chu Richard French Joe Jakubowski Mark Kapoor Don Keener Dan Kelaher Randy Kolvick Josh Miller Thanh Ngo Chuck Stephan From IBM Service and Support Khalid Ansari Brandon Church Become a published author Join us for a two- to six-week residency program! Help write a book dealing with specific - IBM 88743BU | User Manual - Page 18
product acceptance and customer satisfaction. As a bonus, you will develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies - IBM 88743BU | User Manual - Page 19
and scalable upgrade options" on page 9 1.3, "Multinode capabilities" on page 14 1.4, "x3950 M2 Windows Datacenter models" on page 15 1.5, "Integrated virtualization: VMware ESXi" on page 19 1.6, "IBM fourth generation XA-64e chipset" on page 27 1.7, "Processors" on page 33 1.8, "Memory subsystem - IBM 88743BU | User Manual - Page 20
IBM eX4 technology offers a balanced system design with unique scalability, reliability, availability, and performance capabilities to take full advantage of Intel's latest multi-core processors. By connecting four servers together, the single-system image to an IBM System x3950 M2 by adding the - IBM 88743BU | User Manual - Page 21
ports, an operator information panel with power on/off button, and LEDs indicating information such as scalability, network activity, and system errors and warnings. The scalability LED on an x3950 M2 indicates whether the node (building block in a scalable system) is participating in a multinode - IBM 88743BU | User Manual - Page 22
Power-on, Locator and System Error LEDs Gigabit Ethernet 1 Gigabit Ethernet M2 complexes External SAS port Three USB 2.0 ports Integrated dual-port Broadcom Gigabit Ethernet RJ45 ports Remote Supervisor Adapter on page 5, is standard on hypervisor models of the x3850 M2. 4 Planning, Installing, - IBM 88743BU | User Manual - Page 23
connector as viewed from inside the server IBM 4 GB USB Flash Disk pre-loaded with integrated virtualization hypervisor Figure 1-4 On the hypervisor models VMware ESXi 3.5 pre-loaded Standard features for both systems The x3950 M2 and x3850 M2 have the following standard features. We discuss these - IBM 88743BU | User Manual - Page 24
subsystem features include: 4 GB or 8 GB memory standard expandable to 256 GB Support for 1, 2, 4, and 8 GB DDR2 registered DIMMs memory with memory mirroring, hot-add memory with supported operating systems, and Chipkill™. I/O slots and integrated NICs I/O subsystem features include: Seven 64-bit - IBM 88743BU | User Manual - Page 25
Windows Hardware Error Architecture (WHEA) support in the BIOS Trusted Platform Module support. The module is a highly secure start-up process from power-on through to the startup of the operating system boot loader. Advanced Configuration and Power Interface (ACPI) support an integrated standard - IBM 88743BU | User Manual - Page 26
nodes. Figure 1-5 ScaleXpander chip (left); ScaleXpander chip installed on processor board near the front panel of the x3950 M2 (right) 8 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 27
cable (top); cable installed in SMP Expansion Port 1 (bottom) 1.2 Model numbers and scalable upgrade options As discussed previously, the x3850 M2 and x3950 M2 servers are based on IBM eX4 technology. This section lists the available models for each server and where to find more information about - IBM 88743BU | User Manual - Page 28
.ibm.com/xref 1.2.2 x3850 M2 model information The model numbers of the x3850 M2 are listed in Table 1-1. Table 1-1 Models of x3850 M2 Models Description 7141-nRc Standard models of x3850 M2 with dual-core or quad-core Xeon 7200 and Xeon 7300 (Tigerton) processors 7141-3Hc Integrated - IBM 88743BU | User Manual - Page 29
) processors 7233-nSc Standard Models of x3950 M2 with quad-core and six-core Xeon 7400 (Dunnington) processorsa 7141-nAc Datacenter Unlimited Virtualization with High Availability models certified for 32-bit Windows 2003 Datacenter Edition. See 1.4.2, "IBM Datacenter Unlimited Virtualization - IBM 88743BU | User Manual - Page 30
now functionally equal to an x3950 M2. See Figure 1-9 on page 13. Figure 1-7 x3950 M2 enterprise cable management arm 12 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 31
expansion ports 1, 2, and 3 from left to right Route power, Ethernet, fibre cables, video, mouse and keyboard through here Scalability cable brackets to guide the scalability cables from the scalability expansion ports on one node to another node Figure 1-8 x3950 M2 cable management arm mounted on - IBM 88743BU | User Manual - Page 32
The x3950 M2 is the building block that allows supported 8-way, 12-way, and 16-way configurations by adding more x3950 M2s as required. Unlike with the an x3850 M2 that has the ScaleXpander Option Kit installed as described previously in 1.2.4, "Scalable upgrade option for x3850 M2" on page 11 Note - IBM 88743BU | User Manual - Page 33
up to 16 processors, and up to 1 TB RAM installed. Note: At the time of writing, only Windows 2003 Enterprise and Datacenter 64-bit editions, RHEL 5 64-bit, and SLES 10 64-bit support this amount of memory. See 2.6, "Operating system scalability" on page 66 for details. Partitioning Partitioning is - IBM 88743BU | User Manual - Page 34
servers Installation can be performed by IBM, the Business Partner, or the customer. Optional System x Lab Services onsite and IBM Global Services - Remote Support Services can be provided. Table 1-2 on page 11 shows the system models for both 32-bit and 64-bit versions of the operating system. With - IBM 88743BU | User Manual - Page 35
performed by IBM System x Lab Services or IBM Partners certified under the EXAct program. Standard IBM Stage 2 manufacturing integration services can be used. IBM Global Services - Remote Support Services are mandatory. Table 1-2 on page 11 shows the models for both 32-bit and 64-bit versions of the - IBM 88743BU | User Manual - Page 36
Server 2003 Datacenter Edition R2, 32-bit, 1-4 CPUs 4818-NCJ (Japanese) Windows Server 2003 Datacenter Edition R2 x64, 1-4 CPUs (Japanese) 4818-PCJ Note: These upgrade order numbers can only be ordered from the IBM World Wide System x Brand and might not appear in IBM standard configuration tools - IBM 88743BU | User Manual - Page 37
Datacenter Edition R2, 32-bit, 1-4 CPUs (Japanese) Windows Server 2003 Datacenter Edition R2 x64, 1-4 CPUs (Japanese) Order number 4816-NCJ 4816-PCJ Note: These upgrade order numbers can only be ordered from the IBM World Wide System x Brand and might not appear in IBM standard configuration tools - IBM 88743BU | User Manual - Page 38
console has been removed. The function of the service console is replaced by new remote command line interfaces in conjunction with adherence to system management standards. Like the ESX Server, VMware ESXi supports the entire VMware Infrastructure 3 suite of products, including VMFS, Virtual - IBM 88743BU | User Manual - Page 39
specific models of the x3850 M2. These models have an integrated bootable USB flash drive that is securely installed to an internal USB port. With an IBM eX4 server running VMware ESXi (or VMware ESX), applications and services can be deployed in highly reliable and secure virtual machines. Virtual - IBM 88743BU | User Manual - Page 40
Infrastructure Services on file migration VMware ESXi Yes No No No No No No No No VI Foundation VI Standard Yes Yes Yes Yes Yes Yes Yes Yes No Yes No No No No No No No No VI Enterprise Yes Yes Yes Yes Yes Yes Yes Yes Yes 22 Planning, Installing, and Managing the IBM - IBM 88743BU | User Manual - Page 41
DRS with Distributed Power Management (DPM) - VMware HA VMware Infrastructure Standard Edition This edition is designed to bring higher levels of resiliency to IT higher availability, guaranteed service level agreements, and quicker recovery from failures than was previously possible, and comes - IBM 88743BU | User Manual - Page 42
, or number of CPUs of the physical server. However, if you purchase IBM x3850 M2 with VMware ESXi integrated hypervisor and subsequently require additional functionality, you can upgrade ESXi to the VI Enterprise, Standard, or Foundation Editions. See "License upgrades from ESXi to VI3 Editions" on - IBM 88743BU | User Manual - Page 43
M2 includes VMware ESXi V3.5, the embedded virtualization engine on an IBM customized USB Flash Disk. These models include a license for VMware ESXi V3 available in the VMware Infrastructure Enterprise, Standard or Foundation Editions, with a purchase of licenses from IBM as shown in Table 1-8 on - IBM 88743BU | User Manual - Page 44
upgrade, 2-socket,1 or 3 year support Virtual Infrastructure 3, [Enterprise, 1 2 Standard, or Foundation], 2-socket, 1 or 3 year support For details of part numbers, refer to VMware Offerings in the IBM System x Configuration and Options Guide: http://www.ibm.com/systems/xbc/cog/vmwareesx.html - IBM 88743BU | User Manual - Page 45
to resolve this type of problem. Customer implementations that are not covered by an IBM VMware Support Line agreement are required to re-create the problem in a native environment without VMware in order to use Software Maintenance support services for the IBM software that is experiencing the - IBM 88743BU | User Manual - Page 46
DDR2 DDR2 Buffer Buffer DDR2 DDR2 Buffer Buffer DDR2 DDR2 Buffer Buffer CPU 1 CPU 2 CPU 3 CPU 4 Each FSB: 1066 MHz 8.53 GBps IBM X4 Architecture core chipset Memory controller ("Hurricane 4") 8 ports, each: R: 4.26 GBps W: 2.13 GBps Scalability ports 10.24 GBps each HSS-IB 6 GBps HSS-IB - IBM 88743BU | User Manual - Page 47
ports, onboard Ethernet and the standard RSA II. 1.6.1 Hurricane 4 Hurricane 4 is the north bridge component of the IBM eX4 chipset designed for latest and remote latency compared to the X3 chipset in the x3950 Integrated memory controller, NUMA controller, two I/O channels and three scalability - IBM 88743BU | User Manual - Page 48
is minus the portion used for the XceL4v cache. 1.6.3 PCI Express I/O bridge chip Two single-chip PCIe host bridges are designed to support PCIe adapters for IBM x3850 M2 and x3950 M2 servers. The PCIe bridges each have one HSS-IB port that provide connectivity to Hurricane 4 memory controller chip - IBM 88743BU | User Manual - Page 49
System x Servers for Performance, SG24-5287 for more details. 1.6.6 Comparing IBM eX4 to X3 technologies This section discusses the improvements in the design of IBM eX4 technology as compared to the design of previous X3 technology. A block diagram of the X3 technology is shown in Figure 1-13 on - IBM 88743BU | User Manual - Page 50
PCI-X bridge 266 MHz 123456 Six PCI-X 2.0 slots: 64-bit 266 MHz Figure 1-13 Block diagram of IBM X3 x3850/x366 and x3950/x460 IBM eX4 technology builds and improves upon its previous generation X3 technology. The key enhancements are: Processor interface - Quad 1066 MHz front-side bus (FSB), which - IBM 88743BU | User Manual - Page 51
and ability to support external SAS devices through the integrated external SAS port. - Dedicated Southbridge ESI port to support Southbridge devices such 30.72 GBps compared to 19.2 GBps in X3. 1.7 Processors As mentioned previously, the x3850 M2 and x3950 M2 models use one of the following Intel - IBM 88743BU | User Manual - Page 52
All standard models of the x3850 M2 and x3950 M2 have two processors installed. One, two, three, or four processors are supported. Installed 14, you can also connect multiple x3950 M2s to form larger single-image configurations. The processors are accessible from the top of the server after - IBM 88743BU | User Manual - Page 53
cache implements the Advanced Transfer Cache technology. L1 execution trace cache in each core is used to store micro-operations and decoded executable machine instructions. It serves those to the processor at rated speed. This additional level of cache saves decode-time on cache-hits. The Tigerton - IBM 88743BU | User Manual - Page 54
two-way SMP system except that the two processors, or cores, are integrated into one silicon die. This brings the benefits of two-way SMP with supported. Each core has separate L1 instruction and data caches, and separate execution units (integer, floating point, and so on), registers, issue ports - IBM 88743BU | User Manual - Page 55
(formerly known as EM64T) Intel 64 Technology is a 64-bit extension to the industry-standard IA32 32-bit architecture. Intel 64 Technology adds: - A set of new 64-bit general purpose registers (GPRs) - 64-bit instruction pointers - The ability to process data in 64-bit chunks Although the names of - IBM 88743BU | User Manual - Page 56
describe next. To run in compatibility mode, you have to install a 64-bit operating system and 64-bit drivers. When a 64-bit OS and drivers are installed, the processor can support both 32-bit applications and 64-bit applications. The compatibility mode provides the ability to run a 64-bit operating - IBM 88743BU | User Manual - Page 57
x3950 M2 using memory cards, each card has eight DIMM sockets. The server supports up to four memory cards, giving a total of up to 32 DIMM sockets. Some models have two memory cards, others have all four cards as standard. Using 8 GB DIMMs in every socket, the server can hold 256 GB - IBM 88743BU | User Manual - Page 58
the error in the error logs. - If the error is not recoverable, memory scrubbing sends an error message to the light path diagnostics, which then turns on the proper lights and LEDs to guide you to the damaged DIMM. If memory mirroring is enabled, 40 Planning, Installing, and Managing the IBM System - IBM 88743BU | User Manual - Page 59
supporting large amounts of memory, IBM has added the Initialization Scrub Control setting to the BIOS, to let customers on page 118for details. Chipkill memory Chipkill is integrated into the XA-64e chipset, so it does not data around the failed component as previously described. However, if a - IBM 88743BU | User Manual - Page 60
error is detected during POST or memory configuration, the server can automatically disable the failing memory bank and continue operating with reduced memory capacity. You can manually re-enable the memory bank after the problem The x3850 M2 and x3950 M2 support internal RAID-0 and RAID-1. The - IBM 88743BU | User Manual - Page 61
internal to the x3850 M2 and x3950 M2, and all are vacant in the standard models. SAS backplane signal Remote Supervisor Adapter II System Management access Battery Front USB Gbps unidirectional or half duplex) Slots 6 and 7 also support Active PCI hot-swap adapters Chapter 1. Technical overview 43 - IBM 88743BU | User Manual - Page 62
-ROM drive (some models) Serial port 1.11 Networking The IBM x3950 M2 and x3850 M2 servers have an integrated dual 10/100/1000 Ethernet controller that uses the Broadcom NetXtreme II BCM5709C controller. The controller contains two standard IEEE 802.3 Ethernet MACs which can operate in either full - IBM 88743BU | User Manual - Page 63
in 1.8, "Memory subsystem" on page 39. RAID disk arrays are supported for both servers, each with the onboard LSI 1078 for RAID-0 x3850 M2 and x3950 M2 has four internal, hot-swap disk drive bays. The two, standard 1440 W hot-swap power supplies are redundant in all configurations at 220 V. At 110 - IBM 88743BU | User Manual - Page 64
below the other) Two hot-swap redundant 1440 W power supplies Figure 1-20 Redundant memory, fans, and power supplies features 46 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 65
and Active Energy Manager for the IBM x3850 M2 and x3950 M2. out of the rack to diagnose problems, a Light Path Diagnostics panel is the event that maintenance is required, the customer can slide the server out of the Information LED System-error LED NMI button (trained service technician only) - IBM 88743BU | User Manual - Page 66
12.2 BMC service processor The on the Light Path Diagnostics panel to help you diagnose the problem. It also records the error in the BMC system event log. The BMC functions are voltages. The system status is logged and reported to the service processor, if present. System fan speed control The BMC - IBM 88743BU | User Manual - Page 67
, see the x3850 M2 and x3950 M2 User's Guide: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073029 1.12.3 Remote Supervisor Adapter II The x3850 M2 and x3950 M2 have the Remote Supervisor Adapter II service processor as a standard component. This adapter is installed in a dedicated PCI - IBM 88743BU | User Manual - Page 68
the Remote Supervisor Adapter II include: IBM ASIC with integrated PowerPC® 405 core executing at 200 window capture When the operating system hangs, for example, with a blue screen, you might want to capture the window for support purposes. Additionally, the RSA II stores the last failure window - IBM 88743BU | User Manual - Page 69
images of the system where the Web interface of RSA II is running on the remote PC. With this feature, the drives can be made to appear as local drives. See 6.2, "Remote Supervisor Adapter II" on page 316 for more information about the service processor. 1.12.4 Active Energy Manager IBM customers to - IBM 88743BU | User Manual - Page 70
BIOS. The setting, which is disabled by default, assists any OS that has support written into its code to use the security features of this module. The TPM : Disk encryption (For example BitLocker™ Drive Encryption in Windows Server 2008) Digital Rights Management Software license protection and - IBM 88743BU | User Manual - Page 71
by space-saving, rack-optimized servers. This chapters discusses the following topics: 2.1, "Focus market segments and target applications" on page 54 2.2, "Positioning the IBM x3950 M2 and x3850 M2" on page 56 2.3, "Comparing x3850 M2 to x3850" on page 59 2.4, "Comparing x3950 M2 to x3950" on page - IBM 88743BU | User Manual - Page 72
centralizing business computing workloads to reduce ® 2005 and 2008, IBM (DB2®), and Oracle®. Enterprise Resource Planning customer service, and tracking orders. ERP can also include application modules for the finance and human resources aspects of a business. Typically, an ERP system uses or is integrated - IBM 88743BU | User Manual - Page 73
to retailer to consumer. SCM involves coordinating and integrating these flows both within and among companies. applications include the activities of decision-support systems, query and reporting, online -enabling core processes to strengthen customer service operations, streamlining supply chains, - IBM 88743BU | User Manual - Page 74
the rack-optimized two-way servers with increased memory and I/O expansion, higher levels of redundancy, and increased system management features. The IBM eX4 servers are part of the IBM System x scale-up server offering, which is designed to provide: the highest level of processor scalability with - IBM 88743BU | User Manual - Page 75
Servers DS3200/3300/3400 DS4800 High density x3455 BladeCenter E x3100 x3500 X3400 x3200 M2 x3650 BladeCenter H x3550 x3250 BladeCenter S Scale out / distributed computing tended Design Architecture¥ Figure 2-1 The IBM x3950 M2 and x3850 M2 are part of the high-end scale-up portfolio - IBM 88743BU | User Manual - Page 76
from: http://www.redbooks.ibm.com/abstracts/redp4418.html IBM BladeCenter IBM BladeCenter products are designed for complete infrastructure integration, ease of management, energy efficient servers, hardware and software Reliability, Availability, and Serviceability (RAS), and network virtualization - IBM 88743BU | User Manual - Page 77
integration. It demonstrates leadership in power use, high speed I/O, and server density. It provides maximum availability with industry standard components and reduces the number of single points of failure. IBM BladeCenter offers industry's best flexibility and choice in creating customized - IBM 88743BU | User Manual - Page 78
for its three scalability ports and has increased memory throughput with eight high speed memory buffer chips. Furthermore, the x3850 M2 supports more I/O slots from previously having six PCI-X (Tulsa based x3850 has four PCIe and two PCI-X slots) to seven PCIexpress slots. The onboard LSI 1078 - IBM 88743BU | User Manual - Page 79
of the ServeRAID MR10k RAID support Standard not supported only through optional ServeRAID-8i Standard RAID-0 and RAID-1; additional RAID USB Service processor RSA II SlimLine adapter RSA II PCI-X adapter (optional on some models) Embedded virtualization None VMware ESXi integrated hypervisor - IBM 88743BU | User Manual - Page 80
three scalability ports and has increased memory throughput with eight high speed memory buffer chips. Furthermore, the x3950 M2 has support for more I/O slots from previously having six PCI-X slots to seven PCIe slots. The onboard LSI 1078 RAID controller and the optional ServeRAID MR10k installed - IBM 88743BU | User Manual - Page 81
hot-swap bays Four hot-swap bays Disk subsystem Adaptec AIC9410 SAS LSI 1078 SAS RAID support No support standard. RAID support optional with the addition of a ServeRAID 8i adapter Standard RAID-0 and RAID-1, additional RAID features through optional ServeRAID-MR10k PCI-X slots per node Two - IBM 88743BU | User Manual - Page 82
the following issues: Application scalability Operating system scalability Server scalability Storage scalability This section discusses application and operating system scalability. Adding processors can improve server performance under certain circumstances because software instruction execution - IBM 88743BU | User Manual - Page 83
Processor scalability by application type Processors File and Web print server E-mail part of the scalability story. Typically, an important gains from adding processors are only realized when memory is added in parallel. if the operating system does not support more than four processors, or your - IBM 88743BU | User Manual - Page 84
Windows Server 2003 Microsoft Windows Server 2008 and Hyper-V Linux server operating systems In the single chassis 4-way configuration, the IBM eX4 server acts as an industry standard M2 multinode complex helps to address the resource contention issues faced by SMP systems by providing the ability - IBM 88743BU | User Manual - Page 85
it is very important that operating systems have NUMA support. A NUMA-aware operating system must have the application process (which can spawn multiple threads) be serviced from one NUMA node ideally, or from as is the problem with NUMA. But this focus on latency misses the actual problem NUMA is - IBM 88743BU | User Manual - Page 86
to manual oversee the complex task of balancing VMs across multiple NUMA nodes. VMware ESX does provide manual override controls of the system. Several commodity operating systems, such as Windows 2003 Server, provide this level of NUMA support, which is known as initial placement. It might be - IBM 88743BU | User Manual - Page 87
performance and fairness for a datacenter-class system that is expected to support changing workloads with an uptime measured in months or years. To load balancing and page migration To overcome the weaknesses of initial-placement-only systems, as described in the previous section, VMware ESX - IBM 88743BU | User Manual - Page 88
free memory available. The memory migration algorithm also ensures that it IBM x3950 M2. With VMware ESX running on a multinode IBM manually. See Figure 2-4 on page 71. This can be helpful, for example, if a VM runs a memory-intensive workload, such as an in-memory database or a scientific computing - IBM 88743BU | User Manual - Page 89
move a virtual machine to another node after the virtual machine has started running. In this case, the page migration rate of the virtual machine should also be set manually, so that memory from the virtual machine's previous node can be moved to its new node. Chapter 2. Product positioning 71 - IBM 88743BU | User Manual - Page 90
equal shares of the system's resources. You should consider these issues when using manual placement. VMware ESX 3.5 scalability At the time of the in the service console and then reboot the system: # esxcfg-advcfg -k 64 maxPCPUS # esxcfg-boot -b SATA support ESX Server 3.5 supports selected SATA - IBM 88743BU | User Manual - Page 91
. VMware HA At the time of this writing, although ESX 3.5 Update 1 added support for VMware HA feature, it had restrictions--swap space must be enabled on individual NUMA nodes. For a detailed discussion of the features of Windows Server 2003 pertaining to NUMA systems, see the Microsoft document - IBM 88743BU | User Manual - Page 92
2-5 Features of the Windows Server 2003 family Features Standard Edition Enterprise Edition Datacenter Hot-add memory No Yes Yes No NUMA support No Yes Yes No Cluster Service No 1-8 nodes 1-8 nodes No a. At the time of writing, Windows 2003 Datacenter Edition x64 is licensed for up - IBM 88743BU | User Manual - Page 93
://technet.microsoft.com/en-us/magazine/cc194386.aspx Table 2-6 lists features supported by the various Windows Server 2008 editions. Table 2-6 Features of the Windows Server 2008 family Features Standard Edition Enterprise Edition Edition availability 32-bit releasea Yes Yes x64 release - IBM 88743BU | User Manual - Page 94
for IA64 to be used solely on Itanium-based™ hardware platforms. Previously, Windows Server 2003 family had IA64 versions of Windows Server 2003 Standard, Enterprise, and Datacenter Editions that supported running these Windows Server 2003 Editions on the Itanium hardware platform. c. At the time - IBM 88743BU | User Manual - Page 95
Linux 2.6 kernel and components such as the Linux task scheduler and memory allocator, which affect the scaling of the Linux operating system on the IBM x3850 M2 and x3950 M2. Factors affecting Linux performance on a multinode x3950 M2 The overall performance of a NUMA system depends on: The local - IBM 88743BU | User Manual - Page 96
more information, review these Web pages: Red Hat Enterprise Linux Server Version comparison chart http://www.redhat.com/rhel/compare/ IBM ServerProven NOS Support for RedHat Enterprise Linux chart http://www.ibm.com/servers/eserver/serverproven/compat/us/nos/redcha t.html 78 Planning, Installing - IBM 88743BU | User Manual - Page 97
64 GB (theoretical)1 512 GB (certified) 64 TB (theoretical) NUMA support Yes Yes For more information, review the SUSE Linux Enterprise Server 10 Tech major application groups to create integrated end-to-end solutions: Business Intelligence (BI) Customer Relationship Management (CRM) Enterprise - IBM 88743BU | User Manual - Page 98
-bit computing (x64) The combination of Windows Server 2003 for x64 and SQL Server 2005 for x64 offers directly addressable physical memory up to the memory limit of the operating system: 32 GB for Windows Server 2003 Standard Edition, and 80 Planning, Installing, and Managing the IBM System x3950 - IBM 88743BU | User Manual - Page 99
of Windows Server and SQL Server. Several editions of SQL Server 2005 have varying support for x64; however, only these versions are suitable for creating a consolidated SQL Server environment: SQL Server 2005 Enterprise Edition (32-bit and 64-bit) SQL Server 2005 Standard Edition (32-bit and - IBM 88743BU | User Manual - Page 100
. CPU and I/O affinity are used for fine-tuning the allocation of server resources to where they are most required. 82 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 101
instance using soft-NUMA and CPU affinity, so be sure no contention issues arise during the configuration of WSRM. 2.7.2 Microsoft SQL Server 2008 This in SQL Server. Windows Server Resource Manager (WSRM) provided some ability to control processes within Windows by permitting restrictions on - IBM 88743BU | User Manual - Page 102
the capacity of the single system image by adding (in particular) processors, memory, and disk. Scale-out: Adding systems that can be managed and line with business demands. Scale-up systems, such as the IBM eX4 servers, are increasingly being exploited by NUMA-aware virtualization hypervisors - IBM 88743BU | User Manual - Page 103
you can scale-up to systems that support 16 processors (96 cores) and 64 GB of memory on 32-bit versions of operating systems and 1 TB of memory server usually configured equal to primary 2.8.2 Scale-out Scale-out means adding discrete servers to your server farm to gain more processing power. - IBM 88743BU | User Manual - Page 104
availability, load balancing, and high-performance computing. Load balancing is the goal of scaling out. That is to say, we scale-out by adding one or more servers to an existing system be much more expensive than stand-alone versions. 86 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 105
to the fact that as more servers are added to the system, each server manages a system cannot be implemented on shared storage. IBM DB2 and Microsoft SQL Server both implement a It is a leading solution for middle market UNIX®, Windows, and Linux environments. Scale-out capabilities can be extended - IBM 88743BU | User Manual - Page 106
88 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 107
" on page 111 3.3, "Internal drive options and RAID controllers" on page 124 3.4, "Configuring RAID volumes" on page 154 3.5, "PCI Express options" on page 188 © Copyright IBM Corp. 2008. All rights reserved. 89 - IBM 88743BU | User Manual - Page 108
7400 series (Dunnington) 6-core processors Each server can be upgraded to a maximum of four processors. One, two, three, or four processors are supported. Installed processors must be identical in model, speed, and cache size. Figure 3-1 on page 90 shows the locations of the four processors (CPUs - IBM 88743BU | User Manual - Page 109
node. 3.1.1 Processor options Table 3-1 on page 91 lists the processors supported in the x3950 M2 and x3850 M2 (machine type 7141/7144) with geometric center on the topside of the processor integrated heat spreader. For processors without integrated heat spreaders, such as mobile processors, the - IBM 88743BU | User Manual - Page 110
7200 and 7300 Tigerton processors (machine types 7141/7144) do not support the installation of Xeon 7400 Dunnington processors. Processor cache: Tigerton The the processor die: L1 cache The L1 execution, 32 KB instruction and 32 KB data, for data trace cache in each core is used - IBM 88743BU | User Manual - Page 111
of cache on the processor die: L1 cache The L1 execution, 32 KB instruction and 32 KB data, for data trace cache in each core is used to store micro-operations, which are decoded executable machine instructions. It serves those to the processor at rated speed. This additional level - IBM 88743BU | User Manual - Page 112
power up: Processor configuration missmatch error To install the processor options: 1. Turn off the server and peripheral devices. 2. Disconnect the power cords. 3. Wait approximately 20 seconds, then make sure the blue LED light is off. 94 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 113
up to 20 seconds after you remove the power cords. Use this as a guide as to when you can start working in a system. After the blue LED M2 and x3950 M2 with fully opened media hood 6. Review the installation instructions included in the processor option kit. 7. Install the processors in the order - IBM 88743BU | User Manual - Page 114
processor-release lever to the fully opened position, which is approximately a 135° angle. See in Figure 3-5 on page 97. 96 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 115
Lever closed Lever fully opened Figure 3-5 Processor board showing processor release lever 12.Position the processor over the socket, as shown in Figure 3-6, and then carefully press the processor into the socket. Close the processor release lever. Processor Processor orientation indicator - IBM 88743BU | User Manual - Page 116
in the connector next to the processor socket. The VRM and handle are shown in Figure 3-8 on page 99. 98 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 117
purposes. To access the processor settings: 1. Power on or reboot the system. 2. When prompted during system startup, press F1. 3. In the Main Menu window, select Advanced Setup. 4. Select CPU Options. The selection menu in Figure 3-9 on page 100 opens. Chapter 3. Hardware configuration 99 - IBM 88743BU | User Manual - Page 118
. We cover AEM in more detail in 6.4, "Active Energy Manager" on page 334. Active Energy Manager 3.1 is an extension to IBM Director software. A stand-alone version of Active Energy Manager, which runs on top of Embedded Director, is also available. Only the onboard Base Management Controller (BMC - IBM 88743BU | User Manual - Page 119
system, you must configure the BMC IP address through BIOS. This is described in 6.1, "BMC configuration options" on page 300. Note: The IBM performance team observed a drop in performance when power capping is used in a x3950 M2 multinode configuration. The setting is therefore disabled and hidden - IBM 88743BU | User Manual - Page 120
Windows Server 2003, Windows 2008, and how you can identify the operating system power management Linux Various distributions of Linux operating systems, such as SUSE SLES10/SP2 or RHEL5U2 based on kernel 2.6, integrate 1596000 # 102 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 121
. By default, the power scheme Always On is set. This has the effect of running the system at full power regardless of workload demands. Windows includes a power policy named Server Balanced Processor Power and Performance that implements DBS by using the entire range of performance states that are - IBM 88743BU | User Manual - Page 122
in the following folder: %SYSTEMDRIVE%\Windows\Inf %SYSTEMDRIVE%\Windows\Inf For Intel processors, the Intelppm.sys file is used. Because the Intel Xeon is not listed in the support matrix, the Microsoft generic driver Processr.sys is used instead, until the support is available. The power policies - IBM 88743BU | User Manual - Page 123
at a specific state. When you select a new target performance state, Windows Server 2008 chooses the closest match between the current power policy setting and the IBM scalability chip can scale up to a total of eight nodes, IBM supports only up to four nodes. The Hurricane controller supports - IBM 88743BU | User Manual - Page 124
mode Physical mode The sections also discuss the Linux and Windows operating systems regarding clustering. Special mode This mode was created the eX4 technology servers and the older X3 technology servers, if they do not support clustering. Although flat mode is not really a clustering mode, it is a - IBM 88743BU | User Manual - Page 125
Linux distribution, depends on the vendor. The assumption is that the newer IA32E(64T) distributions will support greater than 32 agents. Windows operating systems Windows Server does not require a custom hardware abstraction layer (HAL) because it has been incorporated into 2003 SP1 in order to - IBM 88743BU | User Manual - Page 126
much as 20% depending on the application. For high-performance computing (HPC) applications, we recommend that you enable the Processor Hardware BIOS, if the applications to run on the server have problems with Execution Prevention. For added protection, you might want to enable it, but you should - IBM 88743BU | User Manual - Page 127
Linux OS as Data Execution Prevention (DEP) and Windows OS as Execute Disable Bit (EDB). For IP prefetcher tags the history of each load using the Instruction Pointer (IP) of the load. For each load with disabled by IBM (the opposite of the Intel default). The observations in the IBM performance lab - IBM 88743BU | User Manual - Page 128
errors instruction, that processor core is halted; however, the other processor cores continue normal operation. The processor automatically transitions to a lower core frequency and voltage operating point before entering the Extended HALT state. 110 Planning, Installing, and Managing the IBM - IBM 88743BU | User Manual - Page 129
the Intel Xeon 7200 and 7300 Processor Series Datasheet: http://download.intel.com/design/xeon/datashts/318080.pdf Important: All changes DDR II, registered SDRAM with ECC DIMMs. The supported DIMM options are listed in Table 3-3: Table 3-3 Supported DIMM options for x3850 M2 and x3950 M2 Option - IBM 88743BU | User Manual - Page 130
DIMMs and memory cards as shipped is specified in the server model description. You can obtain details in the latest version of BladeCenter and System x Reference Sheet (xRef) at: http://www.redbooks.ibm.com/xref The DIMMs operate at 533 MHz, to be in sync with the front-side bus. However, the - IBM 88743BU | User Manual - Page 131
CPU 1 CPU 2 Figure 3-12 Memory card location 3.2.2 Memory card The memory card embeds two memory controllers Nova x4, as components of the designated IBM eX4 chipset, to address up to four DIMMs on each memory controller. The memory controller embeds Xcelerated Memory Technology™. You may order - IBM 88743BU | User Manual - Page 132
the memory in all nodes is combined to form a single, coherent physical address space. If you replace the standard pair of DIMMs and install 32x 8 GB DIMMs and four memory cards, both the x3850 M2 and x3950 M2 system is formed, therefore 114 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 133
giving a reduction in overall memory that is available to the operating system of 256 MB per node. The XceL4v architecture is discussed in 1.6.2, "XceL4v dynamic server cache" on page 30. A minimum of 4 GB of memory must be installed in one node if you want to form a multinode complex. This is - IBM 88743BU | User Manual - Page 134
LED DIMM 1 / Error LED DIMM 8 / Error LED Figure 3-14 have the part number 44E4252. Two or four are standard in the x3850 M2 and x3950 M2, depending on about models currently available: http://www.redbooks.ibm.com/Redbooks.nsf/pages/xref Each memory card before adding DIMMs to the next memory - IBM 88743BU | User Manual - Page 135
sequence DIMM pair Memory card 1-4 1 5-8 2 9-12 3 13-16 4 Connector 1/5, 2/6, 3/7, 4/8 1/5, 2/6, 3/7, 4/8 1/5, 2/6, 3/7, 4/8 1/5, 2/6, 3/7, 4/8 - Performance-optimized configuration Section 1.6, "IBM fourth generation XA-64e chipset" on page 27, describes eight independent memory ports - IBM 88743BU | User Manual - Page 136
BIOS, you can hot-swap any memory card if the hot-swap enabled LED is lit. For instructions to hot-swap a memory card, see IBM: System x3850 M2 and System x3950 M2 User's Guide: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073029 After memory-mirroring is enabled, the data that is written to - IBM 88743BU | User Manual - Page 137
run-time scrub engine to ensure memory is good Note: A standard test is still performed across all memory and a run-time system, after memory mirroring is enabled. 3.2.4 Hot-swap memory The x3850 M2 and x3950 M2 support hot-swap memory. If a DIMM fails, it can be replaced with a new DIMM without - IBM 88743BU | User Manual - Page 138
with the service processor, if the lit memory error LED. 3. adding of usable system memory to a running operating system. This is done with an ACPI sequence. Currently, the only operating systems that have this capability and support on the x3850 M2 and x3950 M2 are Windows Server 2003 and Windows - IBM 88743BU | User Manual - Page 139
be added in matched and restrictions, see IBM: System x3850 M2 and System x3950 M2 User's Guide: http://www.ibm.com/support/docview.wss?uid Press F1 during system startup, when prompted. 3. From the Main Menu window, choose Advanced Setup. 4. Select Memory Options. The Memory Settings selection menu - IBM 88743BU | User Manual - Page 140
previously mentioned, hot-add and hot-swap are mutually exclusive. You can enable only one of these features. If you plan to enable hot-add memory and you have a x3850 M2 or x3950 M2 system that comes standard memory that can be hot-added in the future. If and x3950 M2 support Memory ProteXion with - IBM 88743BU | User Manual - Page 141
the run-time hardware memory scrub engine. The hardware memory scrub engine is responsible for checking system memory looking for correctable or uncorrectable memory errors. If you set the scrub rate to default, the chipset is configured to scrub the entire memory array every 24 hours. The setting - IBM 88743BU | User Manual - Page 142
2.5" SAS Hard disk drive backplane ServeRAID MR10k iTBBU w/ custom DDR2 connector Li-Ion Battery Figure 3-16 x3850 M2 and x3950 M2 SAS storage subsystem The LSI 1078 SAS controller operates in the Integrated RAID (IR) mode that supports the internal port by communicating through a x4 SAS cable - IBM 88743BU | User Manual - Page 143
RAID level migration from IR to MR mode: - RAID 1 + RAID 1 → RAID 1 + RAID 1 Two created RAID 1 volumes in the IR (integrated) mode can be imported to the MR ( SAS cable with the hot-swap backplane. The cable is an industry standard MiniSAS 4i cable with SFF-8087 cable and board connectors from the - IBM 88743BU | User Manual - Page 144
the ServeRAID-MR10k installed. Connecting an EXP3000 to the external SAS port with just the onboard LSI 1078 controller is not recommended and is not supported. SAS SFF-8088 external connector on the rear of the server Figure 3-19 x3850 M2 and x3950 M2: external SAS SFF-8088 connector By upgrading - IBM 88743BU | User Manual - Page 145
want to configure, up to 584 GB of disk space can be used internally by using four 146 GB drives. Table 3-10 shows the supported disks. Table 3-10 Supported internal disk options for x3850 M2 and x3950 M2 Part number Description 40K1052 2.5 inch SAS 73 GB 10 K SAS 43X0824 2.5 inch SAS 146 - IBM 88743BU | User Manual - Page 146
-1 (IM) provided by the internal LSI1078 SAS controller and support the external SAS port, install the optional ServeRAID-MR10k controller. in Figure 3-21. Figure 3-21 Optional installed ServeRAID-MR10k The ServeRAID-MR10k supports stripe sizes from 8 KB to 1024 KB. The default stripe size is 128 - IBM 88743BU | User Manual - Page 147
the ServeRAID-MR10k RAID controller, the following RAID levels may be used: RAID-0 Uses striping to provide high data throughput, especially for large files in an environment that does not require fault tolerance. RAID-1 Uses mirroring so that data written to one disk drive is simultaneously written - IBM 88743BU | User Manual - Page 148
which are supported only externally, require at least six hard disk drives. Key features The ServeRAID-MR10k controller has the following features: Logical drive migration You errors and inconsistencies. Patrol read This background operation, also known as data scrubbing, checks for media errors - IBM 88743BU | User Manual - Page 149
integrated chip monitors capacity and other critical battery parameters. It uses a voltage-to-frequency converter with automatic offset error the cache for up to 72 hours, depending on operating environment. IBM recommends that the battery be replaced annually. Replacement part numbers are - IBM 88743BU | User Manual - Page 150
caching. Table 3-12 Customer replacement part numbers Customer replaceable unit (CRU) . A RAID array created with the SAS LSI 1078 can be migrated for use with the ServeRAID-MR10k, but the reverse is not Integrated RAID controller. 132 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 151
IBM: ServeRAID-MR10k User's Guide available from: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5074104 The onboard LSI 1078 and the ServeRAID-MR10k are not supported must install the ServeRAID MR10 driver. Failure to do so is a permanent migration. Installation instructions To install the - IBM 88743BU | User Manual - Page 152
backside of the DIMM. See Figure 3-23. Battery Cable guide Battery cable Cable guide Battery cable connector RAID controller Figure 3-23 Installing the ServeRAID- ServeRAID-MR10k initialization in POST LSI MegaRAID SAS-MFI BIOS Version NT16 (Build Nov 20, 2007) Copyright(c) 2007 LSI Corporation - IBM 88743BU | User Manual - Page 153
HA -0 (Bus 4 Dev 0) IBM ServeRAID-MR10k SAS/SATA Controller FW package: 8.0.1-0029 You have completed the installation of the ServeRAID MR10k card as shown in Figure 3-25 on page 136. The cache has battery backup; the battery is standard with the card. Chapter 3. Hardware configuration 135 - IBM 88743BU | User Manual - Page 154
64 TB LUN 120 devices on each of both external x4 SAS ports. IBM supports using up to 9 cascaded, fully-populated EXP3000 per SAS/SATA connector on 13 ServeRAID-MR10M part number Part number Description 43W4339a IBM ServeRAID-MR10M SAS/SATA Controller a. Kit includes SeveRAID-MR10M, iBBU - IBM 88743BU | User Manual - Page 155
computer system. The backup battery protects cache up to 72 hours, depending on operating environment. IBM recommends that the battery be replaced annually. A damaged or no-recoverable battery can be replaced if you submit a service 3-14 Customer replacement part numbers Customer replaceable unit - IBM 88743BU | User Manual - Page 156
.com/support/docview.wss?uid=psg1MIGR-5076492 Installation instructions To install the ServeRAID-MR10M SAS/SATA controller and the battery package: 1. Mount the iBBU package on the controller board. 2. Prepare the system: Unplug the power cords from the power supplies, disconnect the computer from - IBM 88743BU | User Manual - Page 157
PCIe adapter retention brackets (blue) Rear side of the server Front side of the server Figure 3-26 x3850 M2 and x3950 M2 - PCIe slot location The PCIe slots are beyond the adapter-retention bracket on the right at the rear of the server shown in Figure 3-26 and Figure 3-27 on page 140. Chapter - IBM 88743BU | User Manual - Page 158
slot 6 and slot 7 turn off. 6. Insert the controller in a PCIe slot as shown in Figure 3-28 on page 141 140 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 159
iBBU (Top View) Press Here 85021-06 Edge of Motherboard Edge of Motherboard Figure 3-28 ServeRAID MR10M installation 7. Secure the controller to the computer chassis with the bracket screw. Close the adapter-retention bracket, and use the retention pin as shown in Figure 3-29 on page 142 Note - IBM 88743BU | User Manual - Page 160
in the message in Example 3-8. Example 3-8 ServeRAID-MR10M initialization in POST LSI MegaRAID SAS-MFI BIOS Version NT16 (Build Nov 20, 2007) Copyright(c) 2007 LSI Corporation HA -0 (Bus 84 Dev 0) IBM ServeRAID-MR10M SAS/SATA Controller FW package: 8.0.1-0029 10.Power off the system again. You can - IBM 88743BU | User Manual - Page 161
Figure 3-30 EXP3000 storage expansion unit You can attach the EXP3000 storage expansion unit to the SAS controllers described in the following sections: 3.3.3, "ServeRAID-MR10k RAID controller" on page 128, 3.3.4, "ServeRAID-MR10M SAS/SATA II controller" on page 135 Table 3-15 lists the hard disk - IBM 88743BU | User Manual - Page 162
in Figure 3-31, and the EXP3000 ESM. Table 3-16 EXP3000 Part numbers Part number Description 39R6531 IBM 3m SAS cable 39R6529 IBM 1m SAS cable 39R6515a EXP3000 Environmental Service Module (ESM) a. Second ESM required for attachment to dual controller models of DS3200/DS3400 The cables - IBM 88743BU | User Manual - Page 163
It might be more sensible to chain, for example, two EXP3000 units to one controller and add another controller, which is linked to the second PCIe bridge chip. Figure 1-12 on page 28 shows the slot assignment to the PCIe bridge chips. Installation guidelines The EXP3000 is ready to use after - IBM 88743BU | User Manual - Page 164
Figure 3-32 EXP3000: rear view cabling 4. Turn on the EXP3000 expansion units before or at the same time as you turn on the device that contains the RAID controller. Figure 3-33 on page 147 shows the components of the ESM. 146 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 165
, user guides, and so on. It is located at: http://www.ibm.com/systems/support/x Enter the system Type and Model 172701X in the Quick path field to link to the information for this enclosure. We recommend you also check regularly for new codes and tips. Use the Download and Troubleshoot links - IBM 88743BU | User Manual - Page 166
We used the following steps to update the controller: 1. Download the code release from the following Web address. http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073134 We used the Microsoft Windows package and ran it on a Windows workstation to create bootable diskettes. With these diskettes - IBM 88743BU | User Manual - Page 167
. You might see warning messages stating that this flash is not compatible with all controllers, as shown in Figure 3-35. These are not error messages. * * * SAS Firmware & BIOS Flash Disk * * * NOTE: This utility will scan all LSI 1064, 1068, and 1078 based controllers in the system - IBM 88743BU | User Manual - Page 168
and then flash the ServeRAID controller. 2. Download the most recent firmware: - IBM ServeRAID-MR10k firmware update (Windows) http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073139 - IBM ServeRAID-MR10M firmware update (Windows) http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073389 We - IBM 88743BU | User Manual - Page 169
flash is not compatible with all controllers, as shown in Figure 3-38. These are not error messages. * * * SAS Firmware & BIOS Flash Disk * * * This program will update the firmware on all IBM ServeRAID-MR10k controllers in the system. . Do you wan to continue[Y,N]?_ Figure 3-38 - IBM 88743BU | User Manual - Page 170
internal in the media hood assembly and external in the EXP3000 installed: 1. Download the bootable CD from the following Web address: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-62832 2. Burn a CD or mount the ISO image in the Remote Supervisor Adapter II remote drive of the Web interface - IBM 88743BU | User Manual - Page 171
for the ESM, which fixed the problem of an unexpected reboot of the EXP3000. To update the EXP3000: 1. Download the bootable CD image from the following Web address: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073877 2. Burn a CD or mount the ISO image in the Remote Supervisor Adapter II - IBM 88743BU | User Manual - Page 172
4. Update the ESM modules by clicking the Update button. The update program window is shown in Figure 3-41. Figure 3-41 EXP3000 ESM firmware update Note: Do not power down the system know about to gain access to the SAS storage subsystem. 154 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 173
[ Enabled ] High Precision Event Timer (HPET) [ Enabled ] ɧ ɧ Startup Sequence Options Planar Ethernet PXE/DHCP [ Planar Ethernet 1 ] PCI Device Disk [ Enabled ] Boot on POST/BIOS Error [ Enabled ] Boot Fail Count [ Disabled Watch the window in POST. Press Ctrl+C while - IBM 88743BU | User Manual - Page 174
Logic Corp. MPT SAS BIOS MPTBIOS-6.20.00.00 (2007-12-04) Copyright 2000-2007 LSI Logic Corp. Initializing ..- SLOT ID LUN VENDOR PRODUCT REVISION 0 5 0 IBM-ESXS ST973402SS B522 0 LSILogic SAS1078-IR 1.24.80.00 INT13 SIZE \ NV Boot 68 GB NV 2D:11 Press Ctrl-C to start LSI Corp - IBM 88743BU | User Manual - Page 175
can create two volumes of the same type or a mixture of Integrated Mirroring (IM) or Integrated Striping (IS). Select the RAID level IM or IS. In create this array. Figure 3-47 LSI1078: array creation Figure 3-47 shows a previously created volume in RAID-1 (IM) and one assigned hard disk drive as a - IBM 88743BU | User Manual - Page 176
Slot Num 2 3 ... 2 IBM-ESXST973451SS IBM-ESXST973451SS IBM-ESXST3146855SS B612 B612 RAID Hot Disk Spr [YES] [No] [YES] [No] Drive Pred Size Status Fail (MB 70006 70006 BA26 [No] [Yes 70006 Figure 3-48 RAID-0 (Integrated Mirroring), hot spare You can assign a maximum of 12 hard disk drives - IBM 88743BU | User Manual - Page 177
Version NT16 (Build Nov 20, 2007) Copyright(c) 2007 LSI Corporation HA -0 (Bus 4 Dev 0) IBM WebBIOS will be executed after POST completes You can migrate RAID-0 and RAID-1 volumes that you created WebBIOS The WebBIOS starts up with the initial window, which indicates all available LSI SAS controllers - IBM 88743BU | User Manual - Page 178
in Figure 3-52, the window is separated into three areas: the Controller Function menu, Physical Drives panel, and Virtual Drives panel. Figure 3-52 WebBIOS main window The window also includes icons, as shown in Figure 3-53 on page 161. 160 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 179
3-53 WebBIOS control icons Click to show the application version. Click to turn off sound of onboard controller alarm. Click to exit the application. Click to move to previous window you were viewing. Click to return to the home window. Options in the Controller Function menu are explained in - IBM 88743BU | User Manual - Page 180
3-55, shows the physical specification of the adapter and version for BIOS, firmware package, and summarized information for physical and of completion. Click Next to view the second page of the Adapter Properties window, shown in Figure 3-56 on page 163. On the second page, click IBM System x3950 M2 - IBM 88743BU | User Manual - Page 181
Figure 3-56 Adapter properties (page 2): settings Figure 3-57 Adapter properties (page 3): settings In the second and third pages, change any of the following default settings: Battery Backup This indicates whether you installed a battery backup unit (BBU) on the selected controller. If a BBU is - IBM 88743BU | User Manual - Page 182
the default configurations. Cluster Mode: Disabled is the default This provides additional cluster services while accessing the same data storage. The default setting cannot be changed because it is not supported. Rebuild Rate: 48 is the default This is the percentage of system resources dedicated - IBM 88743BU | User Manual - Page 183
the available PCI ROM space and reduces the chance of getting PCI ROM Allocation errors. Coercion Mode: 1GB-way is the default This forces physical disks of varying (PDF) Self-Monitoring, Analysis and Reporting Technology (S.M.A.R.T.) error. Alarm Control: Disabled is the default This enables, - IBM 88743BU | User Manual - Page 184
that could lead to drive failure, then takes action to correct the errors. The goal is to protect data integrity by detecting physical drive failure before the failure can damage data. IBM calls this data scrubbing. This option indicates the rate for patrol reads for physical drives connected to - IBM 88743BU | User Manual - Page 185
This schedules a consistency check. Click Supported. The Schedule Consistency Check window opens, shown in Figure 3-59. Figure 3-59 Adapter to prevent any impact of data lost in case the amount of disk errors may reach a rate of inconsistency. Consistency checking is a background operation that - IBM 88743BU | User Manual - Page 186
or sequentially check consistency of the virtual drives. Virtual Disks option In the WebBIOS main window, select a particular virtual drive from the list of Virtual Drives and then click Virtual Disks from . Figure 3-60 Virtual disk features 168 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 187
First, however, access all of the features and properties: 1. Select the appropriate virtual disk (VD) from the list (shown in Figure 3-60 on page 168). 2. Select Properties. 3. Click Go. The properties, policies, and operations are displayed, as shown in Figure 3-61. Figure 3-61 Virtual disk - IBM 88743BU | User Manual - Page 188
sure to back up the data before running a consistency check if you think the consistency data might be corrupted. 170 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 189
of the selected virtual disk. Figure 3-63 The WebBIOS virtual disk Policies panel Use this window to specify the following policies for the virtual disk: Access: RW is the default Specify whether in a transaction. This setting is recommended in Standard mode. Chapter 3. Hardware configuration 171 - IBM 88743BU | User Manual - Page 190
is not recommended because the controller is blocked while any outstanding background process have to be completed, before further actions are allowed. Physical drive and Migration panel Options are shown in Figure 3-64 on page 173. 172 Planning, Installing, and Managing the - IBM 88743BU | User Manual - Page 191
disk, you can keep the same number of drives, or you can add drives. You can use the WebBIOS CU to migrate the RAID level of an existing virtual disk. Important: It is strongly recommended that you have a valid backup of any data you want to keep; - IBM 88743BU | User Manual - Page 192
issue I/O operations serially during a reboot. With a RAID-level migration running, a boot can often take more than 15 minutes. After a migration Physical Drives option In the WebBIOS main window (Figure 3-54 on page 161), the software User's Guide does not list this migration option. However we - IBM 88743BU | User Manual - Page 193
Figure 3-65 WebBIOS: physical drives To rebuild by using the extended properties: 1. Select the appropriate physical disk. 2. Select Properties. 3. Click Go. The extended properties about the selected disk are displayed, as shown in Figure 3-65. Figure 3-66 WebBIOS: physical properties The - IBM 88743BU | User Manual - Page 194
Media errors and predicted failure counters SAS address Physical drive state, which can be: - Unconfigured Good - Unconfigured Bad - Online - Offline - Failed - you select undo, the firmware marks this physical disk as unconfigured good. 176 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 195
: Use this command to flash the LED on the selected disk. Unconfigured Bad This state is a physical disk on which the firmware detects an unrecoverable error, the physical disk was Unconfigured Good, or the physical disk could not be initialized. Online You can access at a physical disk if it is in - IBM 88743BU | User Manual - Page 196
state. The drive state changes to rebuild. A status window opens, so you can follow the progress, as failed, but the controller firmware detected an unrecoverable error and marked this disk as failed. Tip: Good. A physical disk previously forced to offline can be marked as missing also. - IBM 88743BU | User Manual - Page 197
clear, create new, or add a virtual disk to any type of array that is supported at the type of installed adapter you selected in the Adapter Selection menu (in Figure 3- 3-54 on page 161. The Configuration Wizard window opens, as shown in Figure 3-71 on page 180. Chapter 3. Hardware configuration 179 - IBM 88743BU | User Manual - Page 198
(Virtual Disk definitions): Define virtual disks using those arrays. Within the virtual disk definition window, you create and adapt the attributes of any virtual disk. 3. (Configuration Preview): Preview clear existing configurations. 180 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 199
New Configuration This option clears the existing configuration and guides you to add new virtual disks. Important: If and their data. The following reminder appears on the window: This is a Destructive Operation! Original configuration and in Figure 3-72: Custom configuration Auto configuration with - IBM 88743BU | User Manual - Page 200
Cancel at any time before the new created configuration is saved. Auto Configuration: with and without redundancy Although Custom Configuration is first in the list on the wizard window, we describe it second. We guide you though the automatic configuration here, but we recommend you use the - IBM 88743BU | User Manual - Page 201
3-73, define the number of disks that you want to add to a virtual disk to assign to a disk group. Figure 3-73 WebBIOS: Configuration Wizard custom DG definition (on left) You can assign all available disks that have the status of Unconfigured good to different disk groups as follow: a. Select the - IBM 88743BU | User Manual - Page 202
Space panel, shown in Figure 3-75, select the disk group for which you want to create a virtual disk. Figure 3-75 WebBIOS: Configuration Wizard custom Span definition Note: The WebBIOS suggests all available RAID levels for disk groups. 184 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 203
you must have two disk groups with the same number of disks and capacity. The selected disk group is added to the Span panel, shown in Figure 3-76. Figure 3-76 WebBIOS: Configuration Wizard custom Span definition on right h. Click Next to change the properties of the virtual disk, as shown in Figure - IBM 88743BU | User Manual - Page 204
when the controller cache has received all the data in a transaction. This setting is recommended in Standard mode. WThru: In Writethrough mode the controller sends a data transfer completion signal to the host a smaller size if you want 186 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 205
to confirm the changes or Reclaim to return to previous settings. j. Click Next when you are finished Configuration after inserting new disks, or by using Custom Configuration if you have unallocated space (a hole Information panel keeps any actions and errors that are reported by the selected - IBM 88743BU | User Manual - Page 206
Express options This section describes the PCI Express (PCIe) subsystem and the supported PCI Express adapters of the x3850 M2 and x3950 M2. The x3850 M2 both have seven half-length full-height PCIe x8 slots, two of which support the hot removal (hot-swap) of adapters. The slots are connected to one - IBM 88743BU | User Manual - Page 207
3. Integrated Ethernet controller 4. PCIe slot 8, 9, 10, 11, 12, 13, 14; then 15, 16, 17, 18, change the sequence and have the server scan one of the first six PCIe slots before it scans the integrated devices. Press F1 when prompted during system POST. In the Start Options panel, set the PCI Device - IBM 88743BU | User Manual - Page 208
do not have to boot from SAN devices, and disabling of onboard devices are also options for solving any PCI ROM allocation errors. Your in country technical support team may advice here. 3.5.4 PCI Express device-related information in the BIOS Information about a PCI Express device can be found in - IBM 88743BU | User Manual - Page 209
hot-plug operation, the device driver does not load. The Windows operating system shows a message similar end cyclic redundancy check) is the data transmission error detection feature. You can override the generation capability might be necessary to avoid issues after changing device power states - IBM 88743BU | User Manual - Page 210
the following Web address: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-62127 Microsoft Windows 2008 supports several PCI Express features natively, such as Active State Power Management (ASPM), Advanced Error Reporting as part of Windows Hardware Error Handling Architecture (WHEA), Extended - IBM 88743BU | User Manual - Page 211
details the ServeRAID-MR10k device information. Scroll to the next/previous PCI device Slot/Device Information Vendor based information ROM usage in assigned correctly. If a system reports an initialization error, by showing a message in POST or in the server error logs. The initial ROM size of each - IBM 88743BU | User Manual - Page 212
completing intensive tests with different operating systems and different external environments, IBM provides all the options that are supported with IBM System x systems, in table form, on the IBM ServerProven Web page: http://www.ibm.com/servers/eserver/serverproven/compat/us/ This list is updated - IBM 88743BU | User Manual - Page 213
of multinode configurations" on page 209 4.7, "Configuring partitions" on page 220 4.8, "Working with partitions" on page 228 4.9, "Observations with scalability configurations" on page 237 © Copyright IBM Corp. 2008. All rights reserved. 195 - IBM 88743BU | User Manual - Page 214
configuration is multiple individual servers connected together by a high speed bus to form a single system image. This system is seen in an installed operating system as one system and the operating system multinode system" on page 234. 196 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 215
partition is started, merges successfully, and is running as a single image. Stand-alone mode A node is booted separately and not merged in more nodes within a multinode complex. It is seen in any type of a supported operating system as one system. Partitioning can be performed at any time to add - IBM 88743BU | User Manual - Page 216
The x3950 M2 is the building block that allows supported 8-way, 12-way, and 16-way configurations by adding other x3950 M2s as required. Note: When Kit installed. All CPUs in every node must be identical. Figure 4-1 Supported multinode configurations Four nodes 8-way or 16-way (Each node is 2-way - IBM 88743BU | User Manual - Page 217
, the next step is to define the single system image, a procedure called partitioning. As a result, all hardware the scalability features. This is different from the previous X3 generation of x3950 systems, which manages the . The BMC handles all instructions and discovers the complex information - IBM 88743BU | User Manual - Page 218
ID (to specific which one is the primary) Partition ID (a unique partition identifier) Node power state Number of partitions 200 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 219
, to check for changes at the scalability port connections, adding new systems and any changes at any existing system in the RSA II Scalability Management menu Web interface if an error occurs. It also indicates static errors, which is a problem of a particular system, or the cabling at a scalability - IBM 88743BU | User Manual - Page 220
an x3850 M2 to an x3950 M2" on page 204 The ScaleXpander key is standard in x3950 M2 model numbers nSn. See 1.2, "Model numbers and scalable upgrade complex, additional cables are required; they are included in the IBM XpandOnDemand Scalability Kit 2, part number 44E4250. Table 4-1 lists part - IBM 88743BU | User Manual - Page 221
after a successful merging is completed. All systems you integrate in a complex must have the same firmware levels components, even if the latest versions are already installed. The UpdateXpress-based your local IBM technical support if the system experiences any hardware-related error. Remember, - IBM 88743BU | User Manual - Page 222
II Time Synchronization [ Enabled ] RSA II Time Zone [ -5:00 Eastern Standard Time ] Figure 4-3 BIOS: Date and Time settings After you install the you want to upgrade, add a scalability partition by ordering the IBM ScaleXpander option kit, part number 44E4249. Review section 4.4, " - IBM 88743BU | User Manual - Page 223
Green: System power LED Blue: Locator LED Amber: System error LED Figure 4-6 x3850 M2 and x3950 M2: rear side LED 3. Remove the cover and the front bezel of x3850 M2 server. See instructions in the System x3850 M2 and x3950 M2 User's Guide. 4. Loosen the captive screws and rotate the media hood - IBM 88743BU | User Manual - Page 224
bezel of the x3850 M2 with the one provided in the ScaleXpander Option Kit. Follow the replacement instructions included in the ScaleXpander Option Kit. Note: The new bezel does not have a sticker where you Settings → RSA LAN interface. 206 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 225
.125 Subnet: 255.255.255.0 You could also set up the RSA IP for each DHCP request. The following IP settings are offered in the BIOS and RSA II Settings: - Try DHCP then use Static IP (default) - DHCP Enabled - Use Static IP Figure 4-8 BIOS: RSA II Settings 4. Configure the RSA II LAN interface - IBM 88743BU | User Manual - Page 226
-node x3950 M2: system code level Description Image release Build level RSA2 v1.02 A3EP26A FPGA v1.22 A3UD22A BMC v3.40 A3BT40A BIOS v1.05 A3E145A Download the following firmware: Remote Supervisor Adapter II firmware: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073124 Field - IBM 88743BU | User Manual - Page 227
version in the update source is the same as that on the system: UpdateXpress CD or UpdateXpress system packs Operating system firmware update packages for Windows the instructions in Rack Installation Instructions for System x3850 M2 and x3950 M2, available from: http://www.ibm.com/support/docview. - IBM 88743BU | User Manual - Page 228
port 3, starting from the left. Each ports has an indicator LED that shows an active connection on this port. 210 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 229
Scalability port 1, 2, and 3 Port 1, 2, and 3 link LED Figure 4-10 x3950 M2: Rear view with scalability connectors Figure 4-11 shows the scalability cable connector. The connectors are the same on both ends of the cable. Figure 4-11 x3950 M2: Scalability cable connector Insert the cables on one - IBM 88743BU | User Manual - Page 230
node 2; then, connect the opposite end of the cable to port 2 of node 2. See Figure 4-13 on page 213. 212 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 231
Node 1 Node 2 Management Network Scalability cabling One RSA LAN connection required for configuration Recommended RSA cabling (optional) Figure 4-13 Two-node x3950 M2: ScaleXpander cable port assignment 3. Route the cables through the Enterprise Cable Management Arms (CMAs) as follows (refer - IBM 88743BU | User Manual - Page 232
1: scalability port 1 → node 2: scalability port 1 3.0 m cable Node 1: scalability port 2 → node 3: scalability port 1 3.0 m cable Node 2: scalability port 2 →node 3: scalability port 2 214 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 233
Node 1 Node 2 Node 3 Management Network Scalability cabling One RSA LAN connection required for configuration Recommended RSA cabling (optional) Figure 4-15 Three-node x3950 M2: ScaleXpander cable port assignment To cable a three-node configuration: 1. Label each end of each ScaleXpander cable - IBM 88743BU | User Manual - Page 234
is included with every the x3950 M2 or the ScaleXpander Option Kit (for upgraded x3850 M2s). The fifth longer (3.26 m) cable is included in the IBM Scalability Upgrade Option 2 Kit. These kits are described in Table 4-1 on page 202. 216 Planning, Installing, and Managing the - IBM 88743BU | User Manual - Page 235
Node 1 Node 2 Node 3 Management Network Node 4 Scalability cabling One RSA LAN connection required for configuration Recommended RSA cabling (optional) Figure 4-16 Four-node x3950 M2: ScaleXpander cable port assignment The scalability cables required for a four-node configuration are listed in - IBM 88743BU | User Manual - Page 236
a short ScaleXpander cable to port 2 on node 2; then, connect the opposite end of the cable to port 2 of node 4. 218 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 237
5. Connect the ScaleXpander cables to node 3: a. Connect one end of the long ScaleXpander cable to port 3 on node 3; then, connect the opposite end of the cable to port 3 of node 4. 6. Repeat step 3 on page 218. 7. Ensure each server can be pulled out fully at the front of the rack and the - IBM 88743BU | User Manual - Page 238
to configure scalable partitions, as shown in Figure 4-18. Figure 4-18 RSA II Web interface with enabled scalability features 220 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 239
The Manage Partition(s) option lists a variety of functions in the Scalable Complex Management panel, shown in Figure 4-18 on page 220. The following buttons enable you to configure, monitor, and manage scalability partitions: Partition Control buttons are shown in Figure 4-19. Figure 4-19 RSA II - IBM 88743BU | User Manual - Page 240
. Partitions Configure buttons are shown in Figure 4-21. Figure 4-21 RSA II Web interface: Partition Configure buttons (create partitions) 222 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 241
The buttons provide the following functions: - Auto partition The Auto button puts all systems in the complex into a single partition. First, power off all systems. Any pre-existing partitions are deleted during power-off. Select a single primary system, then click Auto to create one partition with - IBM 88743BU | User Manual - Page 242
problems Complex and Partition overview Partition ID System serial number Checkmark for the primary server Scalability port: • Grey = no error Cable • Black = good firmware or scalability key • Red = port or cabling error Figure 4-24 RSA II Web interface: complex health The section on the right - IBM 88743BU | User Manual - Page 243
Mul tin ode An entire red row indicates an error with the server as due to firmware or sc al abi li ty System power mo de: • Started = powered on and in POST or OS • Stopped = powered off Partition Status: • Valid = good • Invalid = problems in configuration; no partition Figure 4-25 RSA II Web - IBM 88743BU | User Manual - Page 244
error ports, an unreadable partition descriptor, or other error condition that is unexpected, our tests showed that in the state Invalid or is colored red, an error occurred. Note: An Invalid system that is colored red the systems in the configuration window to match the physical installation - IBM 88743BU | User Manual - Page 245
On merge failure, attempt partial merge?: Yes - Partial merge allowed: Yes Manual partitioning This allows you to assign the servers you want to have in one on page 228. This can only be done at a valid partition. If no error occurs, the view in the middle and right sections change as follows: - The - IBM 88743BU | User Manual - Page 246
in Figure 4-27 on page 229: The selected partition starts immediately. The system's state changes from Stopped to Started. 228 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 247
node. The primary node searches for the secondary node, shown in Example 4-2. Example 4-2 x3950 M2: secondary server merge process IBM BIOS - (c) Copyright IBM Cooperation 2008 Symmetric Multiprocessing System Quad-Core Intel Xeon MP ~2.93GHz Searching for primary server Press BLUE Remind button to - IBM 88743BU | User Manual - Page 248
blue REMIND button After the boxes merged successful you can see the following information about the window on the different boxes: - Primary server display, shown in Example 4-3. Example 4-3 x3950 Standalone mode, followed by a reboot. 230 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 249
Front view: Scalability LED Rear view: Port Link LED Figure 4-29 x3950 M2: Scalability LED and port link LED The scalability port LED becomes solid on in green at the ports where a scalability connection has been established. Reset a partition: To reset a partition, select the Partition ID you - IBM 88743BU | User Manual - Page 250
familiar with the Scalability Manager in the Remote Supervisor Adapter II as described in 4.7, "Configuring partitions" on page 220? 232 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 251
Add a x3950 M2 server system to a complex After the server is installed in the rack and cabled with the scalability cables of each system, you can observe the following behavior: The system can be powered on by one of the following methods: - Use the power (on/off) button on the front of the server - IBM 88743BU | User Manual - Page 252
. See "Start a partition" on page 228. Observe the following behaviors: You can watch the merging process after a partition is started in multinode mode on the window of each member in this partition, see "Start a partition" on page 228. 234 Planning, Installing, and Managing the - IBM 88743BU | User Manual - Page 253
Controller report the timeout and additional error events, at least on the particular affected node. The POST process and the loading of the operating system are shown on the window only at the primary system, as follows: - PCIe devices that are integrated or installed in the other systems - IBM 88743BU | User Manual - Page 254
system. The following behaviors occur if an automatic unexpected reboot (a software or hardware error occurred) occurred without your intervention: All nodes in a partition are affected. The , or defects, and not recoverable errors. 236 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 255
com/systems/support/x 4.9.1 Problem with merging if prerequisites were met If you have problems forming a multinode complex but you have met all prerequisites as described in 4.4, "Prerequisites to create a multinode complex" on page 201, check the error indicators in the Scalable Complex Management - IBM 88743BU | User Manual - Page 256
primary server. I System Complex Powered Up Ensure that no hardware error occurred. See Figure 4-31. In the RSA II Web II Web interface: event log filtering How to resolve the problem To resolve the problem, try the following steps: 1. Power off both systems and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 257
page 201 are not met. Problem 1: CPU mismatch The merging of nodes fails with information about the window, as shown in Example 4-6. to identify the problem To identify the issue: Check that all systems in this complex have two or four CPUs installed. Check that all servers you added to this - IBM 88743BU | User Manual - Page 258
GHz Primarynode has less than 4GB memory installed ...merge not supported 2GB Memory: Installed 4 Processor Packages Installed >> BIOS Version 1.03 - IBM 88743BU | User Manual - Page 259
1 Err SERVPROC 04/16/08, 22:02:03 Resetting system due to an unrecoverable error 2 Err SERVPROC 04/16/08, 22:02:03 Machine check asserted - SPINT, North Bridge Solution and workaround This problem is reported to IBM development. Newer RSA II firmware prevents the serial number of any node from being - IBM 88743BU | User Manual - Page 260
at the same time. They then reboot at the same time, without reporting the error listed in Example 4-8 on page 241. Problem 2: Power state incorrect A node in a configured partition results in becoming of unresponsive 4-35 on page 243. 242 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 261
work around this error by removing all systems in the partitions from the AC power source. Wait 30 seconds and then replug the AC power. We recommend that you regularly check for newer system firmware code and updated product publications such as the Problem Determination and Support Guide and new - IBM 88743BU | User Manual - Page 262
Figure 4-36 IBM System x: technical support on the Web 244 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 263
Chapter 5. Installation This chapter describes the steps involved in installing and configuring supported operating systems on the IBM x3850 M2 and x3950 M2. Firmware and BIOS updates, BIOS settings and operating system support are also discussed as important preliminary tasks prior to beginning the - IBM 88743BU | User Manual - Page 264
systems: You have installed servers and additional components according to instructions in Chapter 3, "Hardware configuration" on page 89 and Chapter that you use the firmware update utility to re-apply the most recent IBM supported firmware level to your node. This ensures that all x3950 M2 nodes - IBM 88743BU | User Manual - Page 265
specific firmware update utilities to a local directory. This directory should provide instructions to update the firmware of the server through bootable media such as a floppy disk image file or a bootable ISO image file. 5.1.3 Performing the updates This section lists the sequence we followed, and - IBM 88743BU | User Manual - Page 266
FPGA firmware This update is in the form of a bootable diskette or ISO image. If you have the RSA II configured and connected to your network, you can use the Remote Console feature to mount a remote diskette IMG file or ISO file, and boot the x3850 M2 or x3950 M2 from the mounted IMG or - IBM 88743BU | User Manual - Page 267
page 249. The FPGA is then reloaded on all nodes in the partition, then each node is powered off and automatically powered on again (this can be 30-second delays in this sequence). Removing any power cable to activate new updated FPGA firmware is not required. Example 5-1 Updating the FPGA code on - IBM 88743BU | User Manual - Page 268
OK. Download to Flash OK. BMC initialization...OK. BMC Firmware and SDRs updated successfully!!! Do you want to clear the SEL (y or n)?y > SEL Cleared. > Flash Loader v1.30.0.54, OSR Technologies. Inc. (c)2007 IPMI Version= firmware 2.0 image 2.0 250 Planning, Installing, and Managing the IBM - IBM 88743BU | User Manual - Page 269
= build Name= 2 32 2 77 A3BT31B 2 32 2 77 A3BT31B Firmware and the image have the same version and build name. Start to program flash? (Y/N) Y_ Start programming... Writing to Address: 0x0007FF80......OK. Download to Flash OK. BMC initialization...OK. BMC Firmware and SDRs updated successfully - IBM 88743BU | User Manual - Page 270
failed. sda : status=0, message=00, host=7, driver=00 sda : sense not available. sda : Write READ CAPACITY failed. sda : status=0, message=00, host=7, driver=00 sda : sense not available. usb 10-6: new high failed. sda : status=0, message=00, host=7, driver=00 sda : sense not available. sda : Write - IBM 88743BU | User Manual - Page 271
sda : status=0, message=00, host=7, driver=00 sda : sense not available. sda : Write Protect is off sda : with upgradable firmware, we recommend you apply the latest versions of firmware on each. This includes: LSI1078 Integrated SAS/SATA RAID controller ServeRAID MR10k SAS/SATA RAID controller - IBM 88743BU | User Manual - Page 272
enter the BIOS Configuration/Setup Utility window, shown in Figure 5-2. Figure 5-2 BIOS Configuration/Setup Utility window 2. Select System Summary to open the System Summary window shown in Figure 5-3. Figure 5-3 System Summary window 254 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 273
cards and capacity, and PCIe adapters installed on the x3850 M2, x3950 M2, or both nodes are correctly detected. 3. From the System Summary window, select Processor Summary → CPUIDs (Figure 5-4) and check that all processors on stand-alone or merged nodes are detected as expected. Figure 5-4 CPUIDs - IBM 88743BU | User Manual - Page 274
In the System Summary window (Figure 5-7), check that the server are detected as shown in the Memory Settings window (Figure 5-8), which also shows memory array, scrub on every (To open the Memory Settings window, select Advanced Setup in the Configuration/Setup Utility window, shown in Figure 5-2 on - IBM 88743BU | User Manual - Page 275
information in the BIOS" on page 190. Figure 5-10 Advanced PCI Settings 5.3 Supported operating systems Before you install your intended operating system, check the IBM ServerProven Web pages, indicated in the following list, to verify that the particular version of Chapter 5. Installation 257 - IBM 88743BU | User Manual - Page 276
an operating system has been tested for compatibility with the x3850 M2 or x3950 M2: IBM ServerProven home http://www.ibm.com/servers/eserver/serverproven/compat/us/index.html IBM ServerProven operating system support matrix http://www.ibm.com/servers/eserver/serverproven/compat/us/nos/matrix .shtml - IBM 88743BU | User Manual - Page 277
-BG. Check Server No Proven x3950 M2 4-node Yes with Update 2 No No No 5.3.2 Windows Server 2003 and 2008 operating systems The ServerProven NOS Support Matrix for Windows is located at: http://www.ibm.com/servers/eserver/serverproven/compat/us/nos/microsoft.html At the time of writing this book - IBM 88743BU | User Manual - Page 278
Server 2008, No No No No Standard x86 Edition Microsoft Windows Server 2008, No No No No Web x86 Edition Windows Server 2003 Microsoft Windows Server 2003 Yes R2 x64 Datacenter Edition Unlimited Virtualization Yes, with SP2 or later Yes, with SP2 or later Planned x3950 M2 4-node - IBM 88743BU | User Manual - Page 279
Server Yes 2003/2003 R2 Standard x86 Edition Microsoft Windows Server No 2003/2003 R2 Web x86 Edition x3950 M2 1-node Yes, with SP2 or later Yes, with SP2 or later Yes, with SP2 or later Yes, with SP2 or later Yes, with SP2 or later No Yes, with SP2 or later Yes, with SP2 or later No x3950 - IBM 88743BU | User Manual - Page 280
M2 4-node No No No No No No No No 5.3.4 SUSE Linux Enterprise Server operating systems The ServerProven NOS support matrix for SUSE Linux Enterprise Server is located at: http://www.ibm.com/servers/eserver/serverproven/compat/us/nos/suseclinux.html At the time of writing this book, the SUSE Linux - IBM 88743BU | User Manual - Page 281
Server 9 for x86 SP3 U2 or later x3950 M2 4-node No No No No No No 5.3.5 Solaris operating systems The ServerProven NOS support matrix for Solaris is located at: http://www.ibm.com/servers/eserver/serverproven/compat/us/nos/suseclinux.html At the time of writing this book, only Solaris 10 was - IBM 88743BU | User Manual - Page 282
hypervisor and customize settings" on page 266 Prerequisites Before you configure ESXi: Ensure you have local or remote (through RSA II Remote Control) video and keyboard console access to the server with the IBM ESXi flash drive installed. Download the following three VMware ESXi guides: - Getting - IBM 88743BU | User Manual - Page 283
- ESX Server 3i Configuration Guide http://www.vmware.com/pdf/vi3_35/esx_3i_e/r35/vi3_35_25_3i_server Disk enabled 3. In Start Options → Startup Sequence Options, set the following startup sequences to IBM Embedded Hypervisor, also shown in Figure 5-12 on page 266: - Primary Startup Sequence: Third - IBM 88743BU | User Manual - Page 284
-ROM and diskette drive 0 in system BIOS 4. Exit the BIOS utility and save you settings when prompted. Boot ESXi hypervisor and customize settings After setting the IBM Embedded Hypervisor to boot as a startup device in the system BIOS, and after POST completes, the server starts to boot VMware ESXi - IBM 88743BU | User Manual - Page 285
and how you interact with the process: 1. After the server POST has completed, the server boots from the IBM USB Flash Disk and loads ESXi hypervisor, shown in Figure 5-13. Figure 5-13 ESXi starting to boot from IBM customized USB Flash Disk with IBM Customized ESXi image Chapter 5. Installation 267 - IBM 88743BU | User Manual - Page 286
and behavior for ESXi is to request a DHCP IP Address. In the event that it cannot find a DHCP server to obtain a DHCP lease, it defaults to the 169.254.0.1/255.255.0.0 IP address and Class B netmask. Figure 5-14 The x3850 M2 is successfully booting the IBM Customized ESXi on the USB Flash Disk, and - IBM 88743BU | User Manual - Page 287
3. In the Customize System window, shown in Figure 5-15, change the default root password to a more secure password in accordance with your root/administrator password policies; select Configure Root Password. Figure 5-15 The Customize System menu when you press F2 at the DCUI ESXi interface 4. Set - IBM 88743BU | User Manual - Page 288
Figure 5-16 Configuring the management network IP address settings 270 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 289
5-17 Options available under Configure Management Network 6. Select IP Configuration and then set the IP Address to match your management network policies (static IP or DHCP). See Figure 5-18 on page 272. Make sure to register the host name in your DNS server (or servers), and set DNS server IP - IBM 88743BU | User Manual - Page 290
performs DNS resolution of your ESXi host name. See Figure 5-19. Figure 5-19 Testing of the management network settings 272 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 291
8. Most of the other settings, typically configured for ESX 3.5, can be also configured similarly by using VI Client, as shown in Figure 5-20. Figure 5-20 Summary tab after connecting to ESXi using VI Client Chapter 5. Installation 273 - IBM 88743BU | User Manual - Page 292
9. ESXi can interface with Common Information Model (CIM) through a set of standard APIs to provide basic processor, memory, storage, temperature, power, and fan hardware-level alerts. 22 on page 275 shows the storage adapters detected. 274 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 293
Figure 5-22 Configuration tab: Detecting Storage Adapters in a x3580 M2 with ESXi 11.ESXi should be able to detect all internal hard drives in the x3850 M2 and any external hard drives connected through the x3850 M2's external SAS connector to EXP3000 enclosures. To confirm this: - Check Storage - IBM 88743BU | User Manual - Page 294
your designated License Server (this is typically configured before or during the installation of VMware VirtualCenter Server). - If you use host-based license files, configure the ESXi host's license source to point to a license file. 276 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 295
VI3 Enterprise, Standard, or Foundation license files, under Licensed Features, you can also enable the appropriate ESX Server Edition (such as Standard) and add- HA) Cluster, you have to enable swap space on the ESXi hosts before adding them to an HA Cluster. To enable Swap: a. In the VirtualCenter - IBM 88743BU | User Manual - Page 296
to a valid directory with sufficient space (for example, 1 GB) to hold the userworld swap file. The userworld swap can be configured on local storage or shared storage. For example, the VMFS box. f. Click OK. g. Reboot the ESXi host. 278 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 297
knowledge base articles from VMware regarding ESXi: "Limited configurations are supported for VMware HA and ESX Server 3i hosts" (KB 1004656) http://kb.vmware.com/kb/1004656 "ESXi 3 Hosts without swap enabled cannot be added to a VMware High Availability Cluster" (KB 1004177) http://kb.vmware - IBM 88743BU | User Manual - Page 298
through its boot process until the Welcome window opens, as shown in Figure 5-27. Figure 5-27 Welcome window for ESXi Installer 6. Press Enter to continue with the Installation. 7. If you accept the End User License Agreement, press F11. 280 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 299
, the direct console window opens and allows you to set up your ESX Server host configuration, as described in "Boot ESXi hypervisor and customize settings" on page 266. For detailed information about installing ESX Server 3i, see the ESX Server 3i Embedded Setup Guide: - IBM 88743BU | User Manual - Page 300
for storage system compatibility in Storage / SAN Compatibility Guide For ESX Server 3.5 and ESX Server 3i (including support for boot from SAN): http://www.vmware.com/pdf/vi35_san_guide.pdf Download the VMware ESX 3.5 installation and configuration guides: http://www.vmware.com/pdf/vi3_35/esx_3/r35 - IBM 88743BU | User Manual - Page 301
3. If you accept the End User License Agreement, click Next. 4. For Partitioning Options, Figure 5-30, select the available local or remote disk (bootable SAN LUN) on which to install ESX 3.5 Update 1. If multiple - IBM 88743BU | User Manual - Page 302
for the default vSwitch and vSwif0 interface for shared Virtual Machine and Service Console access. You may modify the physical Network Adapter that is new users to the ESX 3.5 Udpate 1 system. 10.In the summary window of all the selections you have made during the installation process, confirm the - IBM 88743BU | User Manual - Page 303
and transfer or load the ESX 3.5 Update 1 hypervisor image to the selected disks. 12.Upon completion of the installation, click Finish and reboot the server. The ESX 3.5 Update 1 Hypervisor will boot into the ESX 3.5 startup window with the Service Console IP Address as shown in Figure 5-32. Figure - IBM 88743BU | User Manual - Page 304
Installing Windows Server 2003 x64 on x3950 M2 and x3850 M2 http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5075237 If you are using a ServeRAID MR10k SAS/SATA RAID controller or the onboard LSI 1078 integrated RAID controller, download the latest drivers from: http://www.ibm.com/support/docview - IBM 88743BU | User Manual - Page 305
Server 2003: 1. Boot the server from Windows Server 2003 Installation CD, DVD, or ISO file. 2. Press F6 to load additional storage device drivers when you see the following text: Setup is inspecting your computer's hardware configuration The Windows Setup window opens (see Figure 5-34). Figure 5-34 - IBM 88743BU | User Manual - Page 306
controller. See Figure 5-35. The LSI MegaRAID SAS RAID Controller driver files for the ServeRAID MR10k SAS/SATA Controller should include the following files: - msas2k3.cat - msas2k3.sys - nodev.inf - oemsetup.inf - txtsetup.oem Figure 5-35 Windows Setup detects the LSI MegaRAID SAS RAID Controller - IBM 88743BU | User Manual - Page 307
the server you will be installing onto. Download the appropriate Windows Server 2008 installation instructions: - Installing Windows Server 2008 (32-bit) on x3950 M2 and x3850 M2: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5074895 - Installing Windows Server 2008 x64 on x3950 M2 and x3850 - IBM 88743BU | User Manual - Page 308
boot drives for the IBM ServeRAID MR10k and LSI1078 disk controllers. Install Windows Server 2008 To install Windows Server 2008: 1. Boot the server from Windows Server 2003 Installation CD/DVD or ISO file. 2. Select Install Now. 3. Enter the Product Key for the version of Windows Server 2008 you - IBM 88743BU | User Manual - Page 309
Figure 5-36 Select type of installation: Upgrade or Custom (clean installation) Chapter 5. Installation 291 - IBM 88743BU | User Manual - Page 310
installation process until completed. 8. After Windows Server 2008 boots from the disk partition you selected during the installation process, log on as Administrator and follow the instructions listed in the next section to install additional updated devices drivers for any other devices you might - IBM 88743BU | User Manual - Page 311
II 5709 drivers - Drivers and the latest firmware for additional adapters you might have installed, such as Fibre Channel HBAs, network adapters, and others. If you are installing a 32-bit version of Windows and you have more than 4 GB of RAM installed, add the /PAE switch to the boot.ini file once - IBM 88743BU | User Manual - Page 312
drivers should already be included in your RHEL5 Update 1 installation CD/DVD, or ISO file windows management workstation or a mount point if you are managing the RSA from a linux management workstation) or choose Select File and browse to the CD/DVD, ISO file DVD, CD, or ISO file. 2. When prompted, - IBM 88743BU | User Manual - Page 313
SP1 installation guide: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073088 If you plan to boot from the internal SAS drives or remote storage (for example SAN LUN) and will be installing using the SLES10 Service Pack 1 installation CD/DVD-ROM, you do not have to insert a driver diskette for - IBM 88743BU | User Manual - Page 314
the server from the SUSE Enterprise LInux 10 SP1 Installation CD/DVD, or ISO file. See Figure 5-39. Figure 5-39 SUSE Enterprise Linux Server 10 installer start-up window 2. Select your language and accept the license agreement. 3. Select the type of installation you want to use: new or update. See - IBM 88743BU | User Manual - Page 315
Figure 5-40 SUSE Enterprise Linux 10 Installation Mode window 4. Continue with the installation process. Refer to the installation instructions for additional information. Post-Installation If you are using a 32-bit version of SUSE Enterprise Linux 10, then install the kernel-PAE kernel to access - IBM 88743BU | User Manual - Page 316
298 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 317
the various subsystems, which are implemented to help you manage and service your servers, and also describes embedded features to help you on page 316 6.3, "Use of IBM Director with VMware ESX" on page 334 6.4, "Active Energy Manager" on page 334 6.5, "IBM Director: Implementation of servers" on - IBM 88743BU | User Manual - Page 318
set of standardized and system-specific sensors. The components communicate with the BMC through an embedded Inter-Integrated Circuit (I²C) bus interface. Logged events that are defined as errors are reported, as follows: Into the RSA II event data store Through the BMC driver interface into the - IBM 88743BU | User Manual - Page 319
: Software non-maskable interrupt (NMI) Service processor interrupt (SPINT) routine was started Internal error (IERR) Automatic boot recovery (ABR) Serial redirect Serial over LAN Machine check error capture. Scalability partition management for multinode support Platform event filtering (PEF) and - IBM 88743BU | User Manual - Page 320
x3950 M2 server systems. An in-band communication through the BMC driver interface. Any system management software or utility, which you installed in The BMC is capable of network access through Ethernet port 1. The BMC supports 10/100Mb/s full duplex negotiation. NIC port 1 allows access also to - IBM 88743BU | User Manual - Page 321
, shown in Figure 6-2. Figure 6-2 BIOS: BMC Network Configuration settings menu 4. Enter a host name. 5. Enter the appropriate IP address, subnet mask, and gateway address, or enable DHCP control. 6. Select Save Network Settings in BMC and press Enter. Chapter 6. Management 303 - IBM 88743BU | User Manual - Page 322
server. The logs are defined in different priorities: informal, warning or error event entries, critical or non-critical events. You can access the Analysis (DSA), or IBM Director. An example event log is shown in Figure 6-3. You can select Get Next Entry and Get Previous Entry to page through - IBM 88743BU | User Manual - Page 323
new events. To clear it, select Clear BMC SEL or use tools we mentioned previously. The BMC alerts you if the log reaches 75, 90, or 100 percent full and the privileges level. 6.1.5 Remote control using SMBridge The BMC supports remote control using the OSA SMBridge utility and Serial over LAN. This - IBM 88743BU | User Manual - Page 324
(NMI) signal is received, the processor immediately drops what it was doing and attends to it. The NMI signal is normally used only for critical problem situations, such as serious hardware errors. 306 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 325
error conditions. IBM Technical Support might request you to enable this setting. 6.1.7 BMC firmware update The BMC can be flashed by a bootable image or by Linux or a Windows-based update package. You may download updates from the IBM System x support site: BMC flash update (DOS bootable ISO image - IBM 88743BU | User Manual - Page 326
layer) - Microsoft IPMI driver (any third-party driver must be uninstalled first) 2. IBM IPMI ASR service Download the BMC device drivers To download the drivers appropriate for your server: 1. Go to the Support for IBM System x Web page: http://www.ibm.com/systems/support/x 2. Under Popular links - IBM 88743BU | User Manual - Page 327
x Web page, does not have OSA IPMI driver and layer software available for Linux, Novell, and Windows operating systems to download. For instructions, see the following sections. Install the device drivers on Windows Although the OSA IPMI driver limits the support for single node systems only, the - IBM 88743BU | User Manual - Page 328
mapping layer (library) files in Windows Server 2003 To install the IPMI mapping layer (library) files: 1. Ensure that the OSA IPMI device driver is installed before installing the library software. 2. Download the executable file from the link listed in "Download the BMC device drivers" on page 308 - IBM 88743BU | User Manual - Page 329
so only partially removes the driver, which prevents the installation of a new driver version or re-installation of the previously installed version. If an attempt has already been made to remove the driver using Device Manager, then follow the driver removal instructions listed in this section and - IBM 88743BU | User Manual - Page 330
Server 2003 The Microsoft IPMI driver is not installed by default, however, it is available for manual installation if you have applied Release 2 Service Pack 2, as follows: 1. Click Start → Settings → Control Panel → Add or Remove Programs → Add/Remove Windows Components. The window shown in Figure - IBM 88743BU | User Manual - Page 331
be uninstalled. See Figure 6-11. Ensure that the OSA IBM driver is uninstalled, as described in "OSA IPMI driver uninstall in Windows Server 2003" on page 311. Figure 6-11 Microsoft hardware management notice of third-party drivers 5. Click OK. 6. Click Next. 7. Click Finish. When the installation - IBM 88743BU | User Manual - Page 332
.sys or ipmidrv.sys) and IPMI library files are installed before installing this software. 2. Download the executable, listed in "Download the BMC device drivers" on page 308, and run it. 3. Follow the program's instructions, shown in the window. 4. Reboot the server if the installation procedure - IBM 88743BU | User Manual - Page 333
with Open Source IPMITool Download the latest version of ipmitool from the following location: http://ipmitool.sourceforge.net Note: Problems have been reported with versions of ipmitool prior to v1.8.9. See the following location for details: http://www.ibm.com/support/docview.wss?uid=psg1MIGR - IBM 88743BU | User Manual - Page 334
1. Unzip and untar the downloaded ipmitool package: tar xvzf ipmitool*.tar.gz 2. Configure ipmitool for your used by the BMC Port number Description 623 IPMI communications to SMBridge and IBM Director 664 IPMI communications (secondary) 161 SNMP get and set commands 162 SNMP traps and - IBM 88743BU | User Manual - Page 335
video and networking Video port for the server Service processor daughter card Figure 6-13 Remote Supervisor contains a real-time clock (RTC) timer chip, however the RSA II timer can be synchronized with the system BIOS time or by a Network Time Protocol (NTP) server. The primary user management - IBM 88743BU | User Manual - Page 336
from a remote media such as CD-ROM, ISO files, or floppy images. A connection through Windows Terminal Server can also be established if the remote to access the RSA II Web interface. Remote power control The RSA II supports remote power control to power on, power off, or restart the server with - IBM 88743BU | User Manual - Page 337
The Remote Supervisor Adapter II (RSA II) provides two ways to communicate with system management software: In-band communication through the RSA II driver interface, allowing any system management software or utility that you installed in the operation system to communicate with the RSA II Out-of - IBM 88743BU | User Manual - Page 338
to the service processor port DHCP Control field of the RSA II Settings menu in BIOS: - Try DHCP then Use Static IP (default) - DHCP problem with Linux and its generic human interface device (HID) driver. Linux cannot establish USB communication with the RSA II using the generic HID (which Windows - IBM 88743BU | User Manual - Page 339
described in that section, the adapter is configured by default to look for a DHCP server to obtain an IP address, and if none is available, to use the Advanced Settings menu. To access the RSA II Web interface: 1. Open a supported browser. Enable the use of a Java™ Plugin, if you want to use remote - IBM 88743BU | User Manual - Page 340
. Safeguard your credentials. Losing them requires rebooting the server BIOS and restoring the RSA settings to the factory defaults. After you log on, the welcome window opens. 2. Set the session time-out values as shown in Figure 6-16 on page 323. 322 Planning, Installing, and Managing the - IBM 88743BU | User Manual - Page 341
Figure 6-16 RSA II Web interface Welcome window. The home window of the Web interface opens. See Figure 6-17 on page 324. Chapter 6. Management 323 - IBM 88743BU | User Manual - Page 342
Figure 6-17 RSA II Web interface 6.2.4 Remote console and media To manage servers from a remote location, you often use more than just keyboard-video-mouse (KVM) redirection. For example, for the remote 324 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 343
Using remote media enables USB support after the server is powered up and ISO and diskette image files. Remote media works with the following operating systems: Windows Server 2003 Windows required, which can be downloaded from: http://www.java.com/en/download/manual.jsp Tip: We recommend - IBM 88743BU | User Manual - Page 344
characters and the video speed selector. In the button bar you can access to the Windows Terminal Server if the Remote Desktop Protocol (RDP) is enabled. Each of the buttons pointer). The following keyboard types are supported: US 104-key keyboard Belgian 105-key keyboard 326 Planning, Installing, - IBM 88743BU | User Manual - Page 345
keyboard The Video Speed selector, shown in Figure 6-20 is used to limit the bandwidth that is devoted to the Remote Console display on your computer. Reducing the Video Speed can improve the rate at which the Remote Console display is refreshed by limiting the video data that must be displayed - IBM 88743BU | User Manual - Page 346
the firmware: 1. Download the latest firmware for the RSA II from one of the following pages: - RSA II flash update (PKT files for remote use) http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073124 - RSA II flash update (Windows executable for local use) http://www.ibm.com/support/docview.wss - IBM 88743BU | User Manual - Page 347
driver on the Windows service: 1. Execute the downloaded EXE file on the server with the RSA II. 2. Optionally, click Change to specify an alternate temporary folder for the installation files. The installation process starts automatically after the files are copied. 3. Follow the instructions - IBM 88743BU | User Manual - Page 348
RSA II service in Windows operating system Note: If you have not already done so, change the setting OS USB Selection to Other OS in the system BIOS, as shown in Figure 6-14 on page 320. Linux daemon installation To install the RSA II device driver (daemon) on Linux, ensure that you downloaded the - IBM 88743BU | User Manual - Page 349
two libusb entries (if your version numbers are different, that is okay): libusb-0.1.6-3 libusb-devel-0.1.6-3 Review the appropriate readme file of the RPM Package for prerequisites and installation steps. To install RSA II on RHEL and SLES: 1. Copy the downloaded file to a folder on the Linux - IBM 88743BU | User Manual - Page 350
VMware CD. To install RSA II on VMware: 1. Download or copy the following RSA II daemon package to the VMware server (xx is the version number of the RSA II package): ibm_svc_rsa2_hlp2xxa_linux_32-64.tgz 2. Expand the .tgz file by using the following command: tar xzvf ibm_svc_rsa2_hlp2xxa_linux_32 - IBM 88743BU | User Manual - Page 351
(TCP) - 3389 (default) Windows Terminal Service (TCP) - 6090 (default) TCP IBM Director. If you require management information base (MIB) files, they are on the RSA II firmware ZIP file that also includes the PKT files: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073124 6.2.9 Error - IBM 88743BU | User Manual - Page 352
ASCII text format by clicking the Save Log as Text File button on the bottom of the event log table. Error, Warning, Info), source, and date. The events are displayed in different colors. 6.3 Use of IBM Director with VMware ESX With the Service Update1 of version 5.20.2, the IBM Director supports - IBM 88743BU | User Manual - Page 353
ibm.com/infocenter/eserver/v1r2/index.jsp?topic=/ aem_310/frb0_main.html To download the extension to IBM Director and the stand-alone version, go to: http://www.ibm SUSE Linux Enterprise Server (9 and10) - Windows Server 2003 Enterprise Edition Supported managed hardware: - Rack-mounted server - - IBM 88743BU | User Manual - Page 354
running. When you install AEM, the server starts running. It runs when the IBM Director Server is running. By default, it collects data on the BladeCenter chassis and the Manage Trend Data window in the a graphical user interface (GUI). 336 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 355
request that customizations be deleted during the uninstallation of IBM Director Server Database by using the Manage Trend Data window. Managed systems Active Energy Manager can and provide the captured information by the integrated SNMP out-of-band support. The power capping feature in the BIOS - IBM 88743BU | User Manual - Page 356
capping features. A new Active Energy Manager task icon is added in the IBM Director Management console after installation of the AEM extension (see Figure Manager task icon You use the Active Energy Manager task on the IBM Director Console menu to launch the Active Energy Manager GUI. Use the - IBM 88743BU | User Manual - Page 357
When you install the AEM extension in your IBM Director environment, its task is added to IBM Director Console. Start the Active Energy Manager managed system or for a group of managed systems, the Active Energy Manager window opens and displays a tree of only the selected objects. See Figure 6-27 - IBM 88743BU | User Manual - Page 358
Energy Manager task for those additional managed systems. 6.4.4 Active Energy Manager 3.1 functions This section provides an overview of the functions supported through the Active Energy Manger. The two classes of functions are monitoring functions, which are 340 Planning, Installing, and Managing - IBM 88743BU | User Manual - Page 359
loads when electrical demands or costs are high and, if the system supports power capping, determine appropriate times and levels for power caps (see " top chart shows an example of the power trending information, set at custom intervals. Figure 6-27 on page 340 shows an example of power trending - IBM 88743BU | User Manual - Page 360
raw data provided by the hardware, and converting it into values that are then fed into the control loops. 342 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 361
information. The device in this example shows the ambient and exhaust temperature. Other devices might not available to show an exhaust temperature. iPDU support Section 6.8, "Power Distribution Units (PDU)" on page 357 briefly describes on how iPDUs work with the AEM. An iPDU can record similar - IBM 88743BU | User Manual - Page 362
period and provides a comparison to what would have been consumed had nameplate power been drawn over that entire period. 344 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 363
Figure 6-30 Active Energy Manager: Watt-Hour Meter Management functions The management functions are Power Saver and Power Capping. They provide methods to reduce energy consumption by dropping the processor voltage and frequency. The functions are available for a 90-day trial use. You must purchase - IBM 88743BU | User Manual - Page 364
allows an operator to add extra machines to a data center, which previously had all the data center power allotted to its current systems. Power it by the operator. 6.5 IBM Director: Implementation of servers This section does not discuss the general use of the IBM Director. Rather, it discusses how - IBM 88743BU | User Manual - Page 365
Controller, part number 43W4339 IBM Director version 5.20.2 Service Update 1 - x3850 M2 standard. After installation of the IBM Director Core Services (Level 1) and the IBM Director agent (Level 2), a service processor device driver, and possibly a shared library driver to access the device driver - IBM 88743BU | User Manual - Page 366
it manually. After a system is detected, an icon is added in the IBM Director console. The basic management server saves the addresses of the detected systems in the IBM Director database and supports, based on the x3850 M2 and x3950 M2 server. 6.5.2 Level 0: Implementation by service processors - IBM 88743BU | User Manual - Page 367
systems. The services use standard agent functions, such as detection, authentication, and management. Core Services also installs a Service Location Protocol (SLP) agent, and a Common Information Model Object Manager (CIMOM) with SSL support (at Linux), CIM-classification for WMI (at Windows), and - IBM 88743BU | User Manual - Page 368
services by using the response file: ./dir5.20.2_coreservice_linux.sh -r /directory/response.rsp The directory is the folder, which contains the response file. Windows response file - option indicates: • waitforme (ensure complete installation) 350 Planning, Installing, and Managing the IBM System - IBM 88743BU | User Manual - Page 369
Windows installation program messages) • log=logfile (creates logfile) • verbose (detailed logging) 6. Restart the operating system after the installation is completed. Implementation of Level-1 systems in IBM Director After the IBM Director Core Services are installed and an automatic or manual - IBM 88743BU | User Manual - Page 370
Red Hat Enterprise Linux, version 4.0 Red Hat Enterprise Linux, version 5.0 SUSE Linux Enterprise Server 9 for x86 SUSE Linux Enterprise Server 10 for x86 Microsoft Windows The IBM Director Core Services must be installed before you can install the LSI MegaRAID Provider. Download this extension from - IBM 88743BU | User Manual - Page 371
the RSA II in the operating system" on page 329 describes how to install the device driver in the Linux and Windows operating systems. Installation of the IBM Director agent IBM Director agent is on IBM Director installation CD or you can download it as single installation package from: http://www - IBM 88743BU | User Manual - Page 372
Windows installation of IBM Director agent To install IBM Director agent: 1. Extract the package by using the command: unzip dir5.20.2_agent_windows.zip 2. Change to the extraction folder: cd /install_files_directory/FILES 3. Copy the response file (\directory\FILES\coresvcs.rsp) to another location - IBM 88743BU | User Manual - Page 373
3.5 Embedded and Installable hypervisors offer several management features through integrated CIM and SNMP protocols. The Embedded hypervisor ESX 3i does not have a service console, unlike the Installable version 3.5 has. This means an IBM Director agent cannot be installed to manage this system as - IBM 88743BU | User Manual - Page 374
x3850 M2 and x3950 M2 server systems. This feature must be enabled so it works in IBM Active Energy Manager as described in "Active Energy Manager (power capping)" on page 100. IBM added this feature as a key component of the IBM Cool Blue™ portfolio within Project Big Green. Project Big Green is an - IBM 88743BU | User Manual - Page 375
page 334 explains the use of this application as an IBM Director extension. Power capping is an effective way of Windows 2008 and the Hyper V This allows the usage of efficient multicore systems by running multiple virtual machines in parallel on one system. 6.8 Power Distribution Units (PDU) IBM - IBM 88743BU | User Manual - Page 376
(10A) outlets on front or rear panel Includes hardware to mount in either a standard rack space or side pocket of rack Easily accessible individual breakers per load group for to provide both temperature and humidity values Integrates with IBM Active Energy Manager for consolidated rack power - IBM 88743BU | User Manual - Page 377
PDU and power input cables in your geographical region. Refer to the Rack Power configurator: http://www.ibm.com/systems/xbc/cog/rackpwr/rackpwrconfig.html Table 6-5 IBM Enterprise PDU part numbers Part Feature Description numbera code Outlets Input power C19 C13 71762MX 6090 Ultra Density - IBM 88743BU | User Manual - Page 378
even greater power redundancy. Intelligent PDU Intelligent PDUs (iPDU, with the IBM designation "PDU+") can be remotely managed and have an Ethernet or table form from the iPDU Web interface or by using tools such as IBM Active Energy Manager. They report power and thermal trending also for devices - IBM 88743BU | User Manual - Page 379
this compares to the maximum current that the iPDU can support. Users are alerted when an iPDU approaches its capacity. Note: IBM DPI C13+ and IBM DPI C19 PDU+ PDUs were available before the announcement of Active Energy Manager. Any existing versions of these iPDUs in the field must be upgraded to - IBM 88743BU | User Manual - Page 380
front of the C19 DPI Enterprise PDU+, part number 39M2818. Figure 6-34 IBM Enterprise DPI C19 PDU+ UTG connector, part number: 39M2818 The back of the PDU+ is shared with load group 1, 3, or 5 at the C19 outlets. IBM Ultra Density Enterprise PDU+ Figure 6-35 on page 363 shows the front of the C19 - IBM 88743BU | User Manual - Page 381
UTG inlet Operating mode DIPswitch C13 outlet Ground screw Figure 6-35 IBM Ultra Density Enterprise PDU+, part number: 71762MX The front of the of the PDU+ components The following components are on the iPDUs in the previous figures: Input power connector: Connect a power cord to this connector. - IBM 88743BU | User Manual - Page 382
and to the serial (COM) connector on a workstation or notebook computer and use the workstation or notebook as a configuration console. You can intelligent PDU can be accessed by a Web browser. The section describes the IBM Enterprise DPI C19 PDU+ (part number 39M2818) in particular. The Web - IBM 88743BU | User Manual - Page 383
Figure 6-36 Home page of the IBM Enterprise DPI C19 PDU+ Web interface The iPDU stores the measurement of power data and reports it to a graph, shown in Figure 6-37 on page 366, and presents a chart of power consumption tendencies. Chapter 6. Management 365 - IBM 88743BU | User Manual - Page 384
M2 because the previous tool (PC Doctor) did not meet the requirements for newer systems. DSA Preboot will become the standard embedded diagnostic tool for System x and BladeCenter servers. DSA Preboot is an NVRAM-based version of the of the DSA tool, which is used by the Technical Support teams to - IBM 88743BU | User Manual - Page 385
point panel test CPU and Memory Stress tests IBM light path diagnostics status Service Processor status and configuration Vital product data, Controller configuration Event logs for Service Processors Merged Devices information Memory Diagnostics log DSA Preboot Error log DSA Preboot works with - IBM 88743BU | User Manual - Page 386
(showing all menus) To leave the memory test window use the keyboard's right arrow key to move to the menu Quit and select Quit to DSA. The DSA Preboot is then started. A list of commands is displayed is shown in Figure 6-39. Starting IBM DSA Preboot v1.00 Extracting... Commands: gui - Enter - IBM 88743BU | User Manual - Page 387
how easily DSA Preboot can be updated. Download DSA Preboot from: http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5072294 The DSA can be updated by: Windows operating system Linux operation system (not VMware ESX 3i embedded) Bootable CD or mounted ISO image in the RSA II Web interface USB - IBM 88743BU | User Manual - Page 388
in progress ...DSA key has been flashed successfully Please enter a command. (Type 'help' for commands) usb 5-6: USB disconnect, address 3 > _ 370 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 389
key has been flashed successfully srv-253-217:/tmp/dsa # Update in the Windows operating system To update the DSA Preboot in 32-bit and x64 Windows: 1. Download the following (or similarly named) EXE file to a local disk on your server: ibm_fw_dsa_a3yt71a_windows_32_64.exe Chapter 6. Management 371 - IBM 88743BU | User Manual - Page 390
Double-click the file name. The Update window opens, as shown in Figure 6-40. Figure 6-40 DSA Preboot: the Update process in Windows 3. Select the in the local text viewer. The data and results can be sent to the IBM FTP server to diagnose the information. See "Use of the interactive menu" on - IBM 88743BU | User Manual - Page 391
without creating XML output, by adding option -x. The data collector creates IBM. To transfer data and results to IBM, a network connection is required. You see: Attempted data upload to IBM Network Controllers. Transferring collected data to IBM Service. quit Exits the DSA Preboot menu, - IBM 88743BU | User Manual - Page 392
accessible remote device at the Remote Supervisor remote media window. 2. Use the command copy to save the now available for reviewing by you and by IBM Service. Save the collected information in the operating system the system, you can view the added USB devices in the BIOS. See Figure 6-41. - IBM 88743BU | User Manual - Page 393
adds it as a storage device in the Device Manager (Figure 6-42 on page 375). Figure 6-42 Embedded USB device unhidden in Windows OS In the Linux operating system, shown in Figure 6-43, you see the USB device listed, which can be mounted as storage device. linux-x3850m2-2:~ # - IBM 88743BU | User Manual - Page 394
x3850m2-2:~ # Copy the files with the extension *.xml.gz to a local disk. Provide the log files to the IBM technical support team if requested. After using the command gui. The GUI window opens, shown in Figure 6-44. Figure 6-44 DSA Preboot home window The following sections provide an overview of - IBM 88743BU | User Manual - Page 395
tests at the time of writing this book are in "Use of the interactive menu" on page 373. Figure 6-45 DSA Preboot: Diagnostics window in the GUI After adding tests you select, start the tests by clicking the button Start Tests. The Status column shows the result of this test. For details - IBM 88743BU | User Manual - Page 396
Figure 6-46 DSA Preboot: System Overview information window in the GUI The selections to get information include: of the controller, physical and virtual drive information. Lightpath Snapshot information about error LED states and information about the Light Path Diagnostic (LPD) panel on the - IBM 88743BU | User Manual - Page 397
logs. Note: If your system indicates an error and you are able to boot to the DSA Preboot diagnostics, we recommend you review each of these entries. Help system The embedded Help (Figure 6-47) describes prerequisites, handling instructions, known problems, and provides tips for using the DSA - IBM 88743BU | User Manual - Page 398
380 Planning, Installing, and Managing the IBM System x3950 M2 - IBM 88743BU | User Manual - Page 399
Institute application programming interface Australian Standards American Standard Code for Information Interchange application-specific integrated circuit Advanced System Management Active State Power Management automatic server restart Address Windowing Extensions battery backup unit Boot - IBM 88743BU | User Manual - Page 400
Integrated RAID information store Integrated System Management Processor International Organization for Standards information technology International Technical Support Organization just a bunch of disks kilobyte keyboard video mouse local area network light emitting diode Low-Level Hardware Error - IBM 88743BU | User Manual - Page 401
MSI MSM NIC NMI NMS NOS NTP NUMA NVRAM OCA OEM OLAP OLTP personal computer Peripheral Component Interconnect PCI Express problem determination Error Driver Preboot eXecution Environment Real Application Clusters redundant array of independent disks random access memory remote access services - IBM 88743BU | User Manual - Page 402
machine file system virtual machine manager virtual machine extensions vital product data voltage regulator module Virtual Service Clients Virtual Service Providers Virtualization Technology wide area network Windows Hardware Error Architecture Windows Management Instrumentation Windows System - IBM 88743BU | User Manual - Page 403
Problem Determination and Service Guide http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073027 System x3850 M2 and x3950 M2 Rack Installation Instructions http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073030 IBM ScaleXpander Option Kit Installation Instructions http://www.ibm.com/support - IBM 88743BU | User Manual - Page 404
documentation (installation and user guides) http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5074104 IBM ServeRAID-MR10M documentation (installation and user guides) http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5074105 IBM Information Center: IBM Systems Director Active Energy Manager - IBM 88743BU | User Manual - Page 405
IPMI ASR http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5069505 Download latest Linux Open Source ipmitool utility http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5069538 IBM now supporting Linux open source IPMI driver and utility http://www.ibm.com/support/docview.wss?uid=psg1MIGR - IBM 88743BU | User Manual - Page 406
2003 http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5070012 Intel-based Gigabit Ethernet drivers for Microsoft Windows 2003 and 2008 http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5070807 IBM Remote Supervisor Adapter II Daemon for IA32 Windows http://www.ibm.com/support/docview.wss?uid - IBM 88743BU | User Manual - Page 407
or Integrated RAID SAS driver (Windows) http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073138 ServeRAID MR10k SAS controller firmware update (Windows) http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5073139 ServeRAID MR10M SAS controller firmware update (Windows) http://www.ibm.com/support - IBM 88743BU | User Manual - Page 408
Microsoft See the following Web addresses for more information: Support policy for Microsoft software running in non-Microsoft hardware virtualization software http://support.microsoft.com/kb/897615/ Inside Windows Server 2008 Kernel Changes http://technet.microsoft.com/en-us/magazine/cc194386.aspx - IBM 88743BU | User Manual - Page 409
kb.vmware.com/kb/1004177 Limited configurations are supported for VMware HA and ESX Server 3i hosts http://kb.vmware.com/kb/1004656 The Architecture of VMware ESX Server 3i http://www.vmware.com/files/pdf/ESXServer3i_architecture.pdf I/O Compatibility Guide For ESX Server 3.5 and ESX Server 3i http - IBM 88743BU | User Manual - Page 410
vi3_io_guide.pdf Systems Compatibility Guide For ESX Server 3.0.x http://www.vmware.com/pdf/vi3_systems_guide.pdf VMware Infrastructure 3 Release Notes http://www.vmware.com/support/vi3/doc/vi3_esx3i_e_35u1_vc25u1_rel_no tes.html Tips and Tricks for Implementing Infrastructure Services on ESX Server - IBM 88743BU | User Manual - Page 411
http://people.redhat.com/drepper/cpumemory.pdf Red Hat Enterprise Linux Server Version comparison chart http://www.redhat.com/rhel/compare/ Red Hat Hardware http://trousers.sourceforge.net/ Java Downloads for all operating systems http://www.java.com/en/download/manual.jsp Solaris OS: Hardware - IBM 88743BU | User Manual - Page 412
search for, view, or download Redbooks, Redpapers, Technotes, draft publications and Additional materials, as well as order hardcopy Redbooks, at this Web site: ibm.com/redbooks Help from IBM IBM Support and downloads ibm.com/support IBM Global Services ibm.com/services 394 Planning, Installing, and - IBM 88743BU | User Manual - Page 413
Manager 51, 100, 334-346 BIOS setting 337 capping 346 components 336 console 337 database 337 functions 340 GUI 337 iPDU support 343 managed systems 337 PDU support 343 power capping 346 Power Saver mode 345 power trending 341 sensors 342 server 336 tasks 338 terminology 335 thermal trending 342 - IBM 88743BU | User Manual - Page 414
memory 120 hot-swap memory 119 hypervisor 265 IBM Embedded Hypervisor 265 Initialization Scrub Control 118, 154 Scrub Control 118 System Summary 256 Windows Server 2003 286, 290 BIOS, updating 251 settings 302 configuration 300 connectivity 302 drivers 308 environmental monitoring 301 event log 304 - IBM 88743BU | User Manual - Page 415
130 hot spares 130 internal disks 127 logical drive migration 130 migrating arrays 133 patrol read 130 ports 125 RAID controller 147 features 143 firmware update 153 installation 145 LEDs 147 performance 144 external disk support 6 external SAS port 4, 126, 128 F fans 45 features 2 firmware updates - IBM 88743BU | User Manual - Page 416
spares 130 Grid Computing 58 H IBM Director 349 IPMI driver 309 processors 93 Red Hat Enterprise Linux 293-295 ServeRAID-MR10k 132 ServeRAID-MR10M 137-138 SUSE Linux Enterprise Server 295-297 VMware ESX 281-285 VMware ESXi 264-281 Windows Server 2003 285-289 Windows Server 2008 289-293 integrated - IBM 88743BU | User Manual - Page 417
XceL4v allocation in multinode 114 merge defined 196 failures 239 timeout 197 MIB files 333 Microsoft 80 Microsoft IPMI 308 migration of RAID level 173 models, Windows Datacenter 15 multinode configurations 195-244 adding a server 233 cabling 209 capabilities 198 checklist 232 Complex Descriptor 199 - IBM 88743BU | User Manual - Page 418
VMware ESXi 259 Windows Server 2003 259 Windows Server 2008 259 optical delete a partition 232 manual partitioning 227 power off -30 card installation order 189 hot-plug support 192 initial ROM size 193 installation 138 line cords 359 part numbers 359 PFA error 165 Planar SAS 189 ports Ethernet 4 - IBM 88743BU | User Manual - Page 419
problem determination 124 creating volumes 157 levels 129 migration of RAID levels 125, 173 overview of features 6 ServeRAID-MR10k 128 support 42 rank 31 rear panel 4 address 319 driver 329 error log 333 features 317 firmware update 208, 248, 327 Linux 330 Manage Partition(s) 221 MIB files 333 - IBM 88743BU | User Manual - Page 420
spares 130 IBM Director provider 352 installation 132-133 iTBBU 130 migrating arrays 133 standard memory 6 start a partition 228 storage expansion unit 142 Streams benchmark 109 stripe sizes 128 Supply Chain Management 55 support NUMA 77 RSA II driver 330 scaling 77 support 262 system event log - IBM 88743BU | User Manual - Page 421
69 logical processors 72 manual NUMA controls 70 memory affinity 71 memory migration 69 memory sharing 70 memory support 72 network configuration 284 NUMA controls 70 page migration 69 page sharing 70 partitions 283 prerequisites 281 rebalancer 69 RSA II driver 330 SATA support 72 scalability 68 - IBM 88743BU | User Manual - Page 422
drive count 166 stop on error 166 Stripe Size 185 Unconfigured good 183 virtual disk properties 168-169 virtual drive states 168 wizard 179 WHEA support 7 Windows Datacenter Server 15-19 Windows Server 2003 BIOS settings 286, 290 boot from internal drives 286 driver diskette 288 hot-add memory - IBM 88743BU | User Manual - Page 423
Planning, Installing, and Managing the IBM System x3950 M2 (0.5" spine) 0.475"0.875" 250 459 pages - IBM 88743BU | User Manual - Page 424
- IBM 88743BU | User Manual - Page 425
- IBM 88743BU | User Manual - Page 426
-system image comprising of up to 16 six-core processors, up to 1 TB of high speed memory and support for SUPPORT ORGANIZATION BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers
-
1 -
2 -
3 -
4 -
5 -
6 -
7 -
8
-
9
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
32
-
33
-
34
-
35
-
36
-
37
-
38
-
39
-
40
-
41
-
42
-
43
-
44
-
45
-
46
-
47
-
48
-
49
-
50
-
51
-
52
-
53
-
54
-
55
-
56
-
57
-
58
-
59
-
60
-
61
-
62
-
63
-
64
-
65
-
66
-
67
-
68
-
69
-
70
-
71
-
72
-
73
-
74
-
75
-
76
-
77
-
78
-
79
-
80
-
81
-
82
-
83
-
84
-
85
-
86
-
87
-
88
-
89
-
90
-
91
-
92
-
93
-
94
-
95
-
96
-
97
-
98
-
99
-
100
-
101
-
102
-
103
-
104
-
105
-
106
-
107
-
108
-
109
-
110
-
111
-
112
-
113
-
114
-
115
-
116
-
117
-
118
-
119
-
120
-
121
-
122
-
123
-
124
-
125
-
126
-
127
-
128
-
129
-
130
-
131
-
132
-
133
-
134
-
135
-
136
-
137
-
138
-
139
-
140
-
141
-
142
-
143
-
144
-
145
-
146
-
147
-
148
-
149
-
150
-
151
-
152
-
153
-
154
-
155
-
156
-
157
-
158
-
159
-
160
-
161
-
162
-
163
-
164
-
165
-
166
-
167
-
168
-
169
-
170
-
171
-
172
-
173
-
174
-
175
-
176
-
177
-
178
-
179
-
180
-
181
-
182
-
183
-
184
-
185
-
186
-
187
-
188
-
189
-
190
-
191
-
192
-
193
-
194
-
195
-
196
-
197
-
198
-
199
-
200
-
201
-
202
-
203
-
204
-
205
-
206
-
207
-
208
-
209
-
210
-
211
-
212
-
213
-
214
-
215
-
216
-
217
-
218
-
219
-
220
-
221
-
222
-
223
-
224
-
225
-
226
-
227
-
228
-
229
-
230
-
231
-
232
-
233
-
234
-
235
-
236
-
237
-
238
-
239
-
240
-
241
-
242
-
243
-
244
-
245
-
246
-
247
-
248
-
249
-
250
-
251
-
252
-
253
-
254
-
255
-
256
-
257
-
258
-
259
-
260
-
261
-
262
-
263
-
264
-
265
-
266
-
267
-
268
-
269
-
270
-
271
-
272
-
273
-
274
-
275
-
276
-
277
-
278
-
279
-
280
-
281
-
282
-
283
-
284
-
285
-
286
-
287
-
288
-
289
-
290
-
291
-
292
-
293
-
294
-
295
-
296
-
297
-
298
-
299
-
300
-
301
-
302
-
303
-
304
-
305
-
306
-
307
-
308
-
309
-
310
-
311
-
312
-
313
-
314
-
315
-
316
-
317
-
318
-
319
-
320
-
321
-
322
-
323
-
324
-
325
-
326
-
327
-
328
-
329
-
330
-
331
-
332
-
333
-
334
-
335
-
336
-
337
-
338
-
339
-
340
-
341
-
342
-
343
-
344
-
345
-
346
-
347
-
348
-
349
-
350
-
351
-
352
-
353
-
354
-
355
-
356
-
357
-
358
-
359
-
360
-
361
-
362
-
363
-
364
-
365
-
366
-
367
-
368
-
369
-
370
-
371
-
372
-
373
-
374
-
375
-
376
-
377
-
378
-
379
-
380
-
381
-
382
-
383
-
384
-
385
-
386
-
387
-
388
-
389
-
390
-
391
-
392
-
393
-
394
-
395
-
396
-
397
-
398
-
399
-
400
-
401
-
402
-
403
-
404
-
405
-
406
-
407
-
408
-
409
-
410
-
411
-
412
-
413
-
414
-
415
-
416
-
417
-
418
-
419
-
420
-
421
-
422
-
423
-
424
-
425
-
426
 |
 |
ibm.com
/redbooks
Planning, Installing, and
Managing the
IBM System x3950 M2
David Watts
Jens Reizel
Paul Tan
Kevin Galloway
Understand the IBM System x3950 M2
and IBM x3850 M2
Learn the technical details of
these high-performance servers
See how to configure, install,
manage multinode complexes