HP Xw9400 HP xw9400 Workstation - Optimizing APIC Interrupt Assignments

Mini White Paper

HP Workstations

Optimizing APIC Interrupt Assignments on the xw9400

Introduction:

With the introduction of the xw9400 HP has provided performance sensitive users much greater control over

APIC (Advanced Programmable Interrupt Controller) interrupt assignments.

There was also a significant amount of thought

put into the hardware interrupt line layout to reduce interrupt sharing on the high performance PCIX and PCIE slots, and

high-speed internal interfaces.

Problem:

In some workstation configurations there are more devices that require interrupts then there are available

interrupts to be assigned.

In the xw9400 there are 11 free interrupts available for assignment to 16 potential APIC inputs.

When this happens some devices are forced to share interrupts which can result in sub-optimal performance.

This is partly caused by the need for the Operating System (OS) to call each ISR (Interrupt Service Routine) that is chained to

an interrupt until it finds the device which actually interrupted, and then call all subsequent remaining ISRs until the end of

the ISR chain. Calling an ISR comes at a large time penalty as it requires access to the associated I/O device’s internal

registers which is costly in terms of the number of processor clocks.

As a result, the latency to service the interrupt event is

increased, and the maximum service rate for a given interrupt is decreased. The problem is made worse when the OS

arbitrarily chooses which devices will share interrupts possibly forcing your interrupt sensitive device to the end of an ISR

chain, or to share an ISR chain of devices with very long ISR servicing times.

Some customers find this sharing unacceptable or at the very least undesirable. For instance, Infiniband cards benefit

noticeably from lower latencies when servicing an interrupt event.

xw9400’s Solution:

To solve this problem a new feature has been implemented in the xw9400’s BIOS that allows the user to

control which interrupts an ACPI (Advanced Configuration and Power Interface) based OS will assign to the various APIC

inputs that devices are connected to (see the APIC diagram below).

This allows the customer, not the OS, to choose which

devices share interrupts if sharing is necessary.

To enable this functionality enter the BIOS setup screen by pressing the F10 key when prompted to do so during POST, select

the I/O Menu, choose “APIC Interrupts” and change the setting from “Auto” to “Manual”.

A list of available APIC inputs will

appear, just select which interrupt will get assigned to each.

Please note that some devices are forced to share an interrupt because of how they are physically connected to the APIC

controller.

For example, the 1394 controller and Int A of the PCI 32bit slot (Slot 3) share the same physical interrupt line. This

forces them to always use the same interrupt.

There are no options to manually configure the interrupts for the SAS

controller or devices in slots 4 and 5 because they are connected to a different IOAPIC which has more available interrupts.

Because of this, any interrupt sharing that occurs between

these devices is due to how they are physically connected to

the APIC.

Example:

Let’s say an xw9400 is configured with all the

onboard devices enabled, a 2 function PCI-X Infiniband card

in slot 7, a single function PCI-X card in slot 6,

Graphics in slot

2 and another PCIE card in slot 1.

In total 14 interrupts are being requested; 9 onboard devices,

3 PCI-X functions and 2 PCIE functions.

Since there are only

11 assignable interrupts, 3 interrupts must be shared.

Let’s

assume that the priority is to not share interrupts from the PCI-

X cards, the PCIE cards and SATA0.

To achieve this, the

customer could enter the BIOS Setup by pressing F10, and

assign IRQ5 to LA0 and LAN1, IRQ7 to audio, IRQ10 to USB0

and 1, IRQ16 to SATA0, IRQ17 to SATA1 and 2, IRQ18 to Slot7

IntA, IRQ19 to Slot7 IntB, IRQ20 to Slot6 IntA, IRQ21 to Slot2 intA,

IRQ22 to Slot1 intA and IRQ23 to1394.

This interrupt configuration is represented graphically in the

diagram.

It’s very unlikely that an OS would have chosen

such an optimized interrupt configuration.

Summary:

Customers with demanding high-performance I/O configurations can tune the interrupt capabilities of the

xw9400 to their unique needs and get the last drop of performance out of the system. That’s what the xw9400 was

engineered for, breaking through our customers’ ever more challenging tasks. .

© 2006 Hewlett-Packard Development Company, L.P.

The information contained herein is subject to change without notice. HP shall not be liable for technical or

editorial errors or omissions contained herein.

06/2006