Dell PowerEdge MX840c EMC NVDIMM-N Persistent Memory User Guide
Dell PowerEdge MX840c Manual
View all Dell PowerEdge MX840c manuals
Add to My Manuals
Save this manual to your list of manuals |
Dell PowerEdge MX840c manual content summary:
- Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 1
Dell EMC NVDIMM-N Persistent Memory User Guide May 2020 Rev. A08 - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 2
and tells you how to avoid the problem. WARNING: A WARNING indicates a potential for property damage, personal injury, or death. © 2017 - 2020 Dell Inc. or its subsidiaries. All rights reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 3
Server Hardware Configuration...10 Modular Chassis Hardware Configuration...14 NVDIMM-N Module Details...14 Battery...15 Minimum Platform Firmware ...32 BIOS Requirements...32 Set Up...32 Windows Drivers...32 Storage Class Memory in Windows Server 2016...33 Device manager...33 Identifying the right - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 4
Server 2019...37 NVDIMM-N FW Requirement...37 Driver Architecture Overview...37 New features in Windows Server 2019 Label support Datastore...50 Supported Guest OSes with NVDIMM support...51 Overall Health Status ...51 Operational and Diagnostics Logging information...51 Outdated firmware...52 - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 5
For comparison, the table below provides approximate data access times for DDR4 relative to other server storage media. Table 1. Storage Technology Storage Technology Data Access Time 15K SAS Disk ~ -N basic setup and configuration in a variety of supported operating systems. Introduction 5 - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 6
of document for clarity. Added Modular Server specific information, support for R840, R940xa, changes to the BBU LED behavior and edits to the remainder of document for clarity. NVDIMM-N supported on RHEL 7.5 Added minimum supported platform firmware versions Support for Windows 2019, RHEL 7.6 and - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 7
that allow the NVDIMM-N to persist data are the Controller, Flash, and Power Voltage Regulators that are also integrated on the DIMM. Figure 1. NVDIMM-N Flash • Restore from Flash Normal Operation NVDIMM-Ns are installed in server memory slots. From a hardware perspective, the NVDIMM-Ns appear to - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 8
NVDIMM-N Controller to backup all its DRAM contents to its onboard flash storage. The NVDIMM-N Save event is triggered anytime the server is about to power down and power loss to NVDIMM-Ns is imminent. The backup process takes approximately one minute to complete. This duration is independent of the - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 9
Restore from Flash On server power-up, BIOS re-initializes the DRAM on the NVDIMM-N. BIOS Server OS as Persistent Memory. Note that Persistent Memory is distinct from System Memory which is required for normal OS operation. Please refer to the respective OS Sections for more details on OS support of - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 10
Configuration • NVDIMM-N Module Details • Battery • Minimum Platform Firmware Versions Server Hardware Configuration NVDIMM-Ns are currently supported in the T640, R640,R740/R740XD, R840, R940, R940xa, MX740c and MX840c PowerEdge Servers. Each server supports from 1x to a maximum of 12x 16GB NVDIMM - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 11
Table 3. Supported NVDIMM-N Configurations, 2-Socket (continued) NVDIMM-N NVDIMM-N Capacity RDIMMs RDIMM Capacity 1x 16GB 23x 32GB 736GB 2x 32GB , B7-B12 Note: While other configurations may work, they have not been fully validated and are not currently supported by DellEMC. Hardware 11 - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 12
Figure 6. MX740c Memory Layout Table 4. Supported NVDIMM-N Configurations, R940, MX840c 4-Socket 4-Socket NVDIMM-N NVDIMM-N Capacity RDIMMs RDIMM Capacity 1x 16GB 24x 16GB 384GB 1x 16GB 24x 32GB 768GB 1x 16GB 47x - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 13
Table 4. Supported NVDIMM-N Configurations, R940, MX840c 4-Socket 4-Socket (continued) NVDIMM-N NVDIMM-N Capacity RDIMMs RDIMM Capacity DIMM Population Locations -B6, C1-C12, D1-D12 NVDIMM-Ns: A7-A12, B7-B12 NOTE: 1. NVDIMM-Ns are not supported in CPU Sockets 3 and 4 memory slots Hardware 13 - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 14
support, dual Management Modules provide additional redundancy and robustness to the NVDIMM-N solution. If a user is operating a chassis with NVDIMM-N equipped servers and a Management Module firmware update Operation Green LED Powered Off Off NVDIMM-N going through On power-on and - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 15
board connections for the NVDIMM-N Battery cables in the R740/R740XD (Items 36 and 38). Figure 8 shows how to install and mount the battery in the R740/R740XD. Please refer to each server's Installation and Service Manual for detailed instructions on how to install or remove the battery. Hardware - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 16
Figure 8. R740/R740XD System Board Connections NOTE: Connector locations will be different for each server. Please refer to your particular server's Installation and Service Manual for more information. Figure 9. R740 Battery Installation Instructions NOTE: Battery installation locations will be - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 17
NVDIMM-N modules to be functional on PowerEdge servers, the minimum platform firmware versions are required to be as follows: • BIOS: 1.1.7 • iDRAC: 3.00.00.00 NOTE: Certain operating systems require specific minimum versions of BIOS, NVDIMM-N and/or iDRAC firmware. Please refer to the individual - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 18
focuses only on the BIOS setup options that affect NVDIMM-N operation. For a description of all setup options, please refer to each server's Installation and Service Manual. Persistent Memory BIOS settings are configurable by going into BIOS System Setup. Press F2 at the BIOS screen below to enter - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 19
NUMA (asymmetric) memory configurations. This option is set to Disabled by default. Node interleaving is not supported when NVDMM-N is present in the system. WARNING: When Persistent Memory Interleaving is enabled in the Persistent Memory System Setup page, Node Interleaving setting also - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 20
. Table 7. BIOS setup screen Option Description Node Interleaving Specifies if Non-Uniform Memory Architecture (NUMA) is supported. If this field is set to Enabled, memory interleaving is supported if a symmetric memory configuration is installed. If the field is set to Disabled, the system - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 21
each NVDIMM-N present in the server Table 8. NVDIMM-N Option Description firmware version on the NVDIMM-N. NVDIMM-N Memory Serial Specifies information on the serial number of the NVDIMM-N. Number Remaining Rated Write Endurance [%] Specifies information on the remaining NVDIMM-N flash life - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 22
for each error in the Server System Event Log (SEL) and Life Cycle Log (LCL). Please refer to the JEDEC JESD245B spec for more information on each set to read only mode. Remove input power to the system, reseat the NVDIMM-N module, and restart the server. If the issue persists, replace the faulty - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 23
is operational, and that the server has sufficient power to support this configuration, and that NVDIMM Write-Protect mode has not be enabled in BIOS Setup. UEFI0340 NVDIMM-N Running from Backup Firmware Image at [Location]. Update the NVDIMM-N to the latest firmware. If the issue persists, replace - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 24
Management • NVDIMM-N Error Reporting iDRAC Graphical User Interface The image below shows the iDRAC Web GUI Dashboard when remotely managing the server. Figure 14. iDRAC Graphical User Interface NVDIMM-N Status Select the Memory link on the Dashboard to get more information for memory health - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 25
Battery health. iDRAC will display the NVDIMM-N Battery status only after BIOS has completed the system inventory (at end of POST), and it has detected that NVDIMM-Ns are installed in the server. The Battery status will not be reported if NVDIMM-Ns are not installed. iDRAC NVDIMM-N Management 25 - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 26
a server with NVDIMM-Ns installed is powered down, the server will end of supported life duration. During the next scheduled maintenance, replace the Non-Volatile Dual In-line Memory Module (NVDIMM) device. For more information about the usage of NVDIMMs, see the system Owner's Manual on the support - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 27
shutdown operation or power loss. Remove and reinstall the Non-Volatile Dual In-line Memory Module (NVDIMM) device. If the issue persists, contact your service provider. For information about removing and reinstalling the NVDIMM, see the system Owner's Manual on the support site. MEM9032 The - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 28
The NVDIMM battery is low. Leave the server powered on to allow the NVDIMM-N Battery to charge. If the issue persists, contact your service provider. For information about removing and reinstalling the NVDIMM, see the system Owner's Manual on the support site. BAT0017 The NVDIMM battery has - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 29
save completing, the save will be interrupted and data loss will occur. The server is completely powered down when all LEDs including NVDIMM-N and NVDIMM-N Battery LEDs are off. NOTE: If a Modular server supporting NVDIMM-N is powered on, the entire chassis will be shutdown when there is a full loss - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 30
components will push the power consumption of the chassis above the available power. Additionally, if a user elects to perform a "Virtual Reseat" of an NVDIMM equipped sled, the targeted sled will trigger a save on the NVDIMM-N, but not affect the whole chassis. 30 Server Behavior with NVDIMM-Ns - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 31
must be replaced, all DIMMs must be repopulated in the exact same slot as in the original server. The only migration scenario that is supported is a slot for slot replacement between system boards. Dell EMC also recommends that BIOS NVDIMM settings are identical. DIMM Configuration Changes 31 - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 32
default setting. Windows Server 2019 supports NVDIMM-N interleaving and this option can be set to Enable. 2017-07 Cumulative Update for Windows Server 2016 for x64-based Systems (KB4025334) is required for NVDIMM-N to function correctly in Windows Server 2016. Windows Drivers Windows refers "Storage - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 33
2016 Device manager Below picture describes the windows device manager view of NVDIMM-N root device and NVDIMM-N disk instances in Windows Server 2016. Figure 18. Windows device manager view of NVDIMM-N root device and NVDIMM-N disk instances Identifying the right NVDIMM-N disks Windows PowerShell - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 34
The Serial Number for every NVDIMM-N is unique, and physical location values in PowerShell can be mapped to silk screen using the following table Table 11. DIMM slot location PowerShell PhysicalLocation Dimm Slot Location DIMM Socket 0:Slot 1 A7 DIMM Socket 0:Slot 17 A8 DIMM Socket 0:Slot 33 - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 35
Figure 20. Using device manager GUI Location information in the above can translated to Physical silk screen using following table Table 12. DIMM Slot Location Location DIMM Slot Location 1 A7 17 A8 33 A9 49 A10 65 A11 81 A12 257 B7 273 B8 289 B9 305 B10 321 B11 Windows - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 36
Interface APIs provided by the Windows SCM Disk Driver. This allows existing applications to immediately take advantage if volume in in DAX mode Figure 22. DAX Mode Storage Spaces Support Windows Server 2016 supports NVDIMM-N devices that allow for extremely fast input/output (I/O) operations. - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 37
2019 requires NVDIMM-N devices to support labels in order for the OS to configure namespaces. NVDIMM-N Firmware 8860 doesn't have label space functionality and devices with this firmware cannot be configured by Windows Server 2019. The NVDIMM-N firmware must be updated to FW 9324 or above so that - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 38
created by scmbus.sys, the bus driver for persistent memory. This bus driver object can be found in Device Manager under "System Devices" New features in Windows Server 2019 Label support and Namespace management With Windows Server 2019, OS provides support for Label and Namespace management. When - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 39
Figure 27. List PMEM Unused regions, PMEM Physical Devices and PMEM Disks Figure 28. List/Remove PMEM Disks Windows 39 - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 40
This cmdlet is intended as a "big hammer" recovery mechanism. It is not recommended for normal use. NVDIMM-N Interleaving Windows Server 2019 supports NVDIMM-N interleaving which can further enhance system performance across various applications. This option can be configured through the BIOS Setup - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 41
2016 RTM, a registry entry needs to modified. For details refer to https://support.microsoft.com/en-us/help/4492760/nvdimm-in-read-write-mode-after-backup-power-removed-inwindows-server Windows Errata The following errata affects Windows OSes and are expected to be fixed in a future OS patch. • In - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 42
10 Linux NVDIMM-N hardware is supported on versions 7.3, 7.4 , 7.5 and 7.6 versions of RHEL. Topics: • Identify and Configure PMEM -Persistent Memory Device • Installation • Verify Existing Filesystem • Read-Only Mode NVDIMM-N • Interleave • Management Utility • - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 43
dd command. # dd if=/home/dell/RHEL7.3.iso of=/dev/sdb bs=4M conv=noerror,sync BIOS boots the Linux kernel from USB. Follow the on-screen steps to finish installing RHEL. After the installation is completed, reboot the server. For detailed installation instructions, please refer to https://access - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 44
Copy files to /mnt/nvdimm0 and /mnt/nvdimm1, then shutdown. # echo "writing into nvdimm" >> /mnt/nvdimm0/write.txt # shutdown Interleave Verification Press Power On button on the server When the OS is up running, $ su Verify contents of /mnt/nvdimm0, and do the same thing for /mnt/nvdimm1 # cat /mnt - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 45
dev/pmem0 and /dev/pmem1 # touch /mnt/nvdimm0/write.txt # shutdown Management Utility Management Utility 'ndctl' && mdadm 1. Press PowerOn button on the server. 2. Follow the guidance in Section 4 to setup BIOS. 3. Enable the Persistence Memory, disable "NVDIMM Interleave", disable "NVDIMM Read-Only - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 46
. NVDIMM-N modules that have FW 9324 are not visible in the OS as this firmware has label support. In other words, namespaces do not get enumerated automatically and using ndctl, namespaces have information on how to use ndctl utility refer to the users guide here https://docs.pmem.io/ndctl-users - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 47
access.redhat.com/solutions/3920221 • Red Hat Enterprise Linux 7.6 or later might lose access to the data on NVDIMM-N memory module after updating from NVDIMM-N firmware version 8860 to 9234 or later. To restore access to the data, in Red Hat Enterprise Linux 7.6 CommandLine Interface, login to root - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 48
Firmware requirements Firmware T640, R640, R740, R740XD, R940 Minimum Required Versions R840, R940xa MX740c, MX840c IDRAC 3.15.15.15 3.19.19.19 3.20.20.20 BIOS 1.3.7 1.1.1 1.0.0 NVDIMM-N 9324 9324 9324 Please locate your product on support.dell.com to locate update troubleshooting. - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 49
Translation of ID to physical NVDIMM-N in host system can be seen below. Table 14. Translation of ID to physical NVDIMM-N in host system ID (Hex) 0x1 Decimal 16 Physical Location (L->R) Socket 0 , IMC0, Channel 0 , Slot 1 0x11 17 Socket 0 , IMC0, Channel 1 , Slot 1 0x21 33 Socket 0 , IMC0, - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 50
the fact. To expand the datastore: 1. Migrate your data. 2. Boot into BIOS F2 Setup. 3. Select Persistent Memory as indicated in section 5 of this guide. 4. Select Factory Default and Secure Erase for all NVDIMM-N modules in the system. 5. Follow prompts to reboot and continue on into ESXi. The - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 51
Figure 34. Datastores Supported Guest OSes with NVDIMM support • Windows Server 2016 Build 14393 and above • Windows 10 Anniversary Update Version 1607 and above SEL, and Lifecycle logs for instructions to troubleshoot the NVDIMM-N status. All data loss in the event of power loss 1. Do not shut - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 52
an NVDIMM-N firmware update fails, the DIMM will operate off of backup firmware. For NVDIMM-N modules shipped in 2017 and 2018 this backup firmware is expected any VMs mapped to ANY NVDIMM-N modules in the system to fail to power-on including VMs which are not mapped to the bad NVDIMM-N module. If - Dell PowerEdge MX840c | EMC NVDIMM-N Persistent Memory User Guide - Page 53
at OS run time No OHS error until next boot, logged in SEL real time Invalid Firmware or Firmware running from Backup Slot Maintenance Needed NVDIMM-N Battery Failure at OS run-time All Data Loss as described in the Sections 5 and 6 of this User Guide. ESXi Errata None at this time ESXi 53
Dell EMC NVDIMM-N Persistent Memory
User Guide
May 2020
Rev. A08