Dell PowerEdge External Media System 1434 Improving NFS performance on HPC clu
Dell PowerEdge External Media System 1434 Manual
 |
View all Dell PowerEdge External Media System 1434 manuals
Add to My Manuals
Save this manual to your list of manuals |
Dell PowerEdge External Media System 1434 manual content summary:
- Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 1
Performance on HPC Clusters with Dell Fluid Cache for DAS This Dell technical white paper explains how to improve Network File System I/O performance by using Dell Fluid Cache for Direct Attached Storage in a High Performance Computing Cluster. Garima Kochhar Dell HPC Engineering March 2013, Version - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 2
and its affiliates cannot be responsible for errors or omissions in typography or photography. Dell, the Dell logo, PowerVault, and PowerEdge are trademarks of Dell Inc. Intel and Xeon are registered trademarks of Intel Corporation in the U.S. and other countries. Microsoft, Windows, and Windows - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 3
Fluid Cache for DAS 14 2.5. Tracking solution health and performance 14 2.5.1. Server health and monitoring 14 2.5.2. Dell PowerEdge Express Flash PCIe SSD health and monitoring 14 2.5.3. Dell Fluid Cache for DAS health and monitoring 16 3. Performance ...16 3.1. Sequential writes and reads 17 - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 4
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Tables Table 1. Table 2. Table 3. Table 4. Table 5. NFS server and storage hardware configuration 8 NFS server software and firmware configuration 9 Hardware configuration for - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 5
paper describes how to improve I/O performance in such a NFS storage solution with the use of Dell Fluid Cache for DAS (DFC) technology. It describes the solution and presents cluster-level measured a how-to recipe in the Appendix that provides step-by-step instructions on building the solution. 5 - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 6
instructions on how to configure such a solution and provide information on the benchmarks and tests that were run for this study complete this document. 1.1. Dell Fluid Cache for DAS (direct-attached storage) DFC is a write-back host caching software. DFC combines multiple Dell PowerEdge™ Express - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 7
NFS Performance on HPC Clusters with Dell Fluid Cache for DAS sections provide details on each of these components as well as information on tuning and monitoring the solution. 2.1. NFS storage solution (baseline) The baseline in this study is an NFS configuration. One PowerEdge R720 is used as the - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 8
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Figure 2. NFS server Table 1. NFS server and storage hardware configuration Server configuration NFS SERVER PowerEdge R720 PROCESSORS Dual Intel(R) Xeon(R) CPU E5-2680 @ 2.70 GHz MEMORY 128 GB. 16 * 8 GB 1600MT/s RDIMMs - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 9
disks are combined using Linux Logical Volume Manager (LVM). DFC does not support caching of an LVM device, hence a single PERC was used for Detailed instructions on configuring this storage solution are provided in Appendix A: Step-by-step configuration of Dell Fluid Cache for NFS. 2.2. Dell Fluid - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 10
. Detailed instructions on configuring DFC for this storage solution are provided in Appendix A: Step-by-step configuration of Dell Fluid Cache benchmark the capabilities of the solution. Using the latest quarter height Dell PowerEdge M420 blade server5 as the building block for the I/O cluster, - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 11
Dell Fluid Cache for DAS Table 5. I/O cluster details I/O cluster configuration CLIENTS CHASSIS CONFIGURATION INFINIBAND FABRIC For I/O traffic 64 PowerEdge M420 blade servers 32 blades in each of two PowerEdge M1000e chassis Two PowerEdge and management Each PowerEdge M1000e chassis has - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 12
performance. These options were selected based on extensive studies done by the Dell HPC team. Results of these studies and the tradeoffs of the tuning the optimizations applied to the storage solution. Detailed instructions on configuring this storage solution are provided in Appendix A: Step- - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 13
some important XFS bugs6 and is recommended for this solution. • The PowerEdge R720 is a dual-socket system that uses the Intel Xeon E5-2600 file system as well. The options used for this solution are similar to the Dell NSS and are noatime, allocsize=1g, nobarrier, inode64, logbsize=262144, attr2. - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 14
recommends that OMSA's server and storage management components be installed on the NFS server in this solution. OMSA v7.1.2 provides support for DFC. 2.5.2. Dell PowerEdge Express Flash PCIe SSD health and monitoring Configuring the PCIe SSDs in the NFS server is a very straightforward task. The - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 15
warranty is 3 years, 25 PBW. The health of the device can be monitored using Dell OMSA utilities. OMSA reports the SSD "Device Life Remaining" and "Failure Predicted". " Device 0:2:0 : Ready : Not Applicable : PCIe : SSD : 100% : No : B1490208 : 1.3.7 : DELL_P320h-MTFDGAL350SAH 15 - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 16
a component under the storage sub-section of the OMSA GUI. /opt/dell/fluidcache/bin/fldcstat is a command-line utility that provides extensive statistics of the IO, etc. Additional details are available in the DFC User's Guide in [3]. 3. Performance This section presents the results of performance - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 17
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS • DFC in Write-Back mode (DFC-WB) - This configuration builds on the baseline by adding DFC as described in Section 2.2, and - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 18
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Throughput in MiB/s Figure 5. Large sequential write performance Sequential writes 2500 2000 1500 1000 500 0 1 2 4 8 16 32 48 64 Number of concurrent clients baseline DFC-WB DFC-WT Throughput in MiB/s Figure - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 19
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS better than the baseline since the data is already in the DFC cache. As expected on read operations, WB and WT - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 20
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS IOPS Figure 8. Random read performance 140000 120000 100000 80000 60000 40000 20000 0 1 Random reads 2 4 8 16 32 48 64 Number of concurrent - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 21
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Number of create() ops per second 45000 40000 35000 30000 25000 20000 15000 10000 5000 0 Figure 9. Metadata file create performance File - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 22
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Number of remove() ops per second 50000 45000 40000 35000 30000 25000 20000 15000 10000 5000 0 Figure 11. Metadata file remove - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 23
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Figure 12 shows that on a cold- 12. Cold-cache sequential reads Cold-cache Sequential Reads Throughput in MiB/s 3500 3000 2500 2000 1500 1000 500 0 1 2 4 8 16 32 48 64 Number of concurrent clients DFC-WB DFC-WB- - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 24
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Figure 13 shows that on a cold-cache read for the random tests, peak IOPS of the DFC configurations drop from ~123, - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 25
DAS User's Guide at www.dell.com/support http://www.dell.com/us/enterprise/p/poweredge-r720/pd 4. NFS tuning and the optimized Dell NFS Storage Solutions (NSS) Dell™ HPC NFS Storage Solution Dell HPC NFS Storage Solution High Availability (NSS-HA) Configurations with Dell PowerEdge 12th Generation - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 26
-management/w/wiki/1760.openmanageserver-administrator-omsa.aspx 8. Dell PowerEdge Express Flash PCIe SSD www.dell.com/poweredge/expressflash http://support.dell.com/support/edocs/storage/Storlink/PCIe%20SSD/UG/en/index.htm http://content.dell.com/us/en/home/d/solutions/limited-hardware-warranties - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 27
Dell Fluid Cache for NFS This appendix provides detailed step-by-step instructions on the configuration of the storage solution described in this white paper. Readers familiar with Dell -attached external storage, and the Dell Fluid Cache for DAS (DFC) software. A PowerEdge R720 is used as the NFS - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 28
This will be used for swap space. 2. Install Red Hat Enterprise Linux (RHEL 6.3) on the RAID 1 virtual disk of the PowerEdge R720. To use any GUI features of DFC or Dell OpenManage, select the "Desktop" group of packages. 3. This solution is based on the 2.6.32-279.14.1.el6.x86_64 errata kernel for - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 29
Management" components of Dell OpenManage Server Administrator (OMSA) v7.1.2 on the PowerEdge R720. Note that only v7.1.2 supports DFC at the time backup of /etc/sysconfig/nfs and change the number of threads. Set the NFS service to start on boot. # cp /etc/sysconfig/nfs{,.orig} # sed -i 's/#RPCNFSDCOUNT=8/ - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 30
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS 2. Change the OS I/O scheduler to "deadline". To the end of the kernel line in /etc/grub.conf for the .14.1 errata - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 31
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Partitions Hot Spare This disk will be used as the NFS storage. All the commands in this section are executed on the PowerEdge R720 NFS server. 1. Note the controller ID of the PERC H810 adapter. From the output below, the - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 32
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Status Name Slot ID State Firmware Version Minimum Required Firmware Version Driver Version Minimum Required Driver Version Storport Driver Version Minimum - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 33
sdc 2. Set up the cache pool for Dell Fluid Cache. Verify that the DFC service is running. [root@nfs-dfc ~]# service fluid_cache status Add two SSDs to the cache etc/exports /home/xfs *(rw,no_root_squash,sync) 5. Restart the NFS service. Now the clients can mount the NFS share over the ib0 interface. - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 34
Check the fldcstat manual pages for options and descriptions of the statistics that are available. 5. Dell OpenManage Server Administrator off the firewall before implementing this alternative: [root@nfs-dfc ~]# service iptables stop; chkconfig iptables off A.7. Performance tuning on clients 1. - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 35
system on the server. • Mount XFS file system on the server and start the NFS service. • Mount NFS Share on clients. In addition for the cold cache tests described in Section 3.4, the disk managed by Dell Fluid Cache for DAS was disabled and the SSDs that are part of the cache pool - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 36
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS IOzone Argument -t +m -w -I -O Description Number of threads Location of clients to run IOzone on when in clustered mode Does not unlink (delete) - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 37
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS B.2. mdtest mdtest can be downloaded from http:// during the metadata testing: • Unmount NFS share on clients. • Stop the NFS service and umount the XFS file system on the server. • Mount XFS file system on the server and start - Dell PowerEdge External Media System 1434 | Improving NFS performance on HPC clu - Page 38
Improving NFS Performance on HPC Clusters with Dell Fluid Cache for DAS Metadata file and directory creation test: # mpirun -np 32 -rr --hostfile ./hosts /nfs/share/mdtest -d /nfs/share/filedir -i 6 -b 320 -z 1 -L -I 3000 -y -u -t -C Metadata
-
1 -
2 -
3 -
4 -
5 -
6 -
7 -
8
-
9
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
32
-
33
-
34
-
35
-
36
-
37
-
38
 |
 |
Garima Kochhar
Dell HPC Engineering
March 2013, Version 1.0
Improving NFS Performance on HPC
Clusters with Dell Fluid Cache for DAS
This Dell technical white paper explains how to improve Network File
System I/O performance by using Dell Fluid Cache for Direct Attached
Storage in a High Performance Computing Cluster.