Dell VNX5600 VNX2 Deduplication and Compression - Maximizing effective capacit
Dell VNX5600 Manual
View all Dell VNX5600 manuals
Add to My Manuals
Save this manual to your list of manuals |
Dell VNX5600 manual content summary:
- Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 1
White Paper EMC VNX2 Deduplication and Compression VNX5200, VNX5400, VNX5600, VNX5800, VNX7600, & VNX8000 Maximizing effective capacity utilization Abstract This white paper discusses the capacity optimization technologies delivered in the EMC® VNX™2 series of - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 2
Copyright © 2016 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate of its publication date. The information is subject to change without notice. The information in this publication is provided "as is." EMC Corporation makes no representations or - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 3
Servers 22 Common Data Environment 22 Performance and Capacity Savings 22 VNX Block Deduplication for VNX File 23 Requirements and Supported Configurations 23 Management ...25 Considerations ...32 VNX Block Compression 33 VNX Block Compression Overview 33 The VNX Block Compression Process 33 - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 4
Data Mover level deduplication and compression settings 47 File System level deduplication and compression settings 48 Viewing Deduplication Statistics 49 VNX File Deduplication and Compression commands 50 Conclusion ...50 References ...51 Appendix A: NTFS File System Conditioning with SDelete - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 5
on, the system intelligently manages capacity-optimization processes as new data is written. With Unisphere®, users can manage block and file data compression features discussed in this paper are available on the VNX2 Series (VNX5200, VNX5400, VNX5600, VNX5800, VNX7600, and VNX8000). As part of EMC - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 6
VNX Block Deduplication VNX Block Deduplication is a software feature included with the VNX2 series. In general, deduplication is the process of identifying duplicate data contained within a set of block storage objects and consolidating it such that only one actual copy of the data is used by many - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 7
Figure 2. VNX Block Deduplication example Figure 3 shows the process of deduplicating data on LUNs 1, 2, and 3. The first instance of each 8 KB chunk of data is kept in the pool, while the duplicates are deleted. Each LUN containing the chunks of data that were deduplicated will point to the copies - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 8
In the above example, the deduplication process removes the duplicates of the original of 8 KB chunks. Consider a storage array which stores multiple copies of the same data. In this scenario, the array needlessly copies much of the data repeatedly. Deduplication consolidates most of the 8KB - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 9
enabled, but the Allocation Owner will change to match the owner of the deduplication container. A LUN's Allocation Owner can be viewed on the General tab within the LUN Properties window in Unisphere. This is also where you determine which SP owns the pool's deduplication container by viewing the - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 10
container cannot be changed without disabling deduplication on all LUNs within a pool. Figure 5. Deduplication Container Balance Example VNX Block Deduplication runs in the background on the contents of a deduplication container post process, 12 hours after the previous deduplication run completed - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 11
slice may be freed back to the pool for use when the slice no longer contains any data. If a deduplication process has run on a pool for 4 hours straight, the run is paused and the system checks for deduplication processes waiting to run on that SP. If a deduplication process is waiting to run, it - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 12
the new layout and the performance enhancements the new layout provides, existing LUNs must be migrated into a new deduplication container. Management You can easily manage VNX Block Deduplication by using either the Unisphere software or NaviSecCLI at the System, Pool, and LUN levels. Once enabled - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 13
, or the array is running an earlier version of VNX OE for Block. If running on a code previous to 05.33.000.5.072, you must manually set the Default Owner of the LUN to the SP that owns the deduplication container within the pool. Figure 7: Create LUN Advanced tab EMC VNX2 - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 14
If the user changes the Default Owner to the peer SP in VNX OE for Block version 05.33.000.5.072 and later, the message shown in Figure 8 is displayed. By changing the Default Owner, the user is choosing to avoid the optimal ownership model for the LUN. As the message indicates, changing the owner - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 15
Figure 10. Right-click LUN method to turn on deduplication In VNX OE for Block version 05.33.008.5.119, the Deduplication tab within the LUN Properties dialog box has changed with the addition of Block Deduplication for VNX File. In this release and later, the user will no longer be able to enable - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 16
of SPB. After enabling deduplication on an existing LUN, the user should verify that the Default Owner and Current Owner match the Allocation Owner of the LUN. If the Default Owner does not match the Allocation Owner, the Default Owner should be changed by using the Properties page of the LUN in - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 17
Shortly after deduplication is enabled on a LUN, the LUN state will show as enabling in the Deduplication tab in the LUN properties window. While the LUN State shows as enabling, the LUN is migrating into the deduplication container within the pool. At any time, the user can view deduplication - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 18
the deduplication container. As with other pool-based LUNs, the default setting is Start High then Auto-Tier. If the FAST VP enabler is not installed on the system, the tiering policy will be a fixed value, and will reflect the Initial Tier data placement policy of the first deduplicated LUN on - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 19
on each of these commands and others, please consult the VNX Command Line Interface Reference for Block documentation, located on EMC Online Support. Block Deduplication with Multicore FAST Cache, FAST VP, and Multicore Cache The basic purpose of VNX Block Deduplication is to eliminate data - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 20
After duplicate data has been removed within a deduplication container, FAST VP operations become more efficient as there is less data to track and promote. Also, with a consolidated storage footprint, FAST VP can better use space on the pool. Consider multiple LUNs on the same pool that have been - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 21
LUN that is already in the midst of migrating). Similarly, a LUN that is currently migrating cannot have its deduplication-enabled state changed. VNX Snapshots Fully supported. Cannot take a snapshot while the LUN is being migrated in or out of the pool (when the LUN is in "enabling" or "disabling - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 22
of the workload is served out of Flash, the system utilizes more CPU resources to maximize that potential. Block Deduplication is a data service that requires additional overhead to the normal code path. When considering Block Deduplication, it is recommended to review the Utilization of the system - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 23
File resource. If these requirements are not met, a warning appears and deduplication is not enabled. Enabling VNX Block Deduplication is only supported on Mapped Storage Pools or Meta Volume based File Systems and Checkpoint Storage. The underlying block storage for these configurations must be - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 24
Example. When choosing to extend a Meta Volume or File Pool in which deduplication is enabled, only expanding the resource with whole disk volumes is supported. When the extension is started, the new LUN will inherit the deduplication state of the Meta Volume or File Pool. In this instance, VNX - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 25
Management VNX Block Deduplication is typically enabled on Block Storage Pool LUNs directly via Unisphere or NaviSecCLI. In VNX OE for Block version 05.33.008.5. - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 26
Figure 18. Mapped Storage Pool Properties Window. After enabling or disabling VNX Block Deduplication, a migration into or out of the deduplication container begins. To view which LUNs are migrating, view the LUNs within the Block Storage Pool, as shown in Figure 19. If the Deduplication column is - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 27
Deduplicated LUN SP Owner within the Deduplication tab of the Storage Pool Properties window. Once the optimal owner is determined, the user must manually change the Default Owner, shown in Figure 12, on each of the affected LUNs and issue nas_storage -check -all and nas_storage - failback commands - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 28
Figure 21. Could not enabled VNX Block Deduplication error. Figure 22 below shows an error seen while deduplication is enabling or disabling on the underlying LUNs. The error informs the user that not all LUNs have a deduplication state which matches the setting on VNX File. Listed in this error are - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 29
example was previously creating using two disk volumes. Using the Control Station CLI command nas_fs -create with the -fixed_block_dedup option allows the user to specify if block deduplication is enabled or disabled when creating a File System on a Meta Volume. Figure 24. File System on Meta Volume - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 30
. When VNX Block Deduplication is enabled on the Checkpoint Storage, Auto Extend is disabled. To extend the Checkpoint Storage the user must manually extend it using a Meta Volume utilizing whole disk volumes. If the Checkpoint Storage is created from a Storage Pool, FixedBlock Deduplication Enabled - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 31
Figure 27. shows the Enter File System Info step within the New File System Wizard. When using the wizard to create a File System on a Mapped Storage Pool, the user will be presented with the state of Fixed-Block Deduplication for the Mapped Storage Pool. As VNX Block Deduplication is enabled at the - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 32
Figure 29 shows the Deduplication tab within the Block Storage Pool Properties window. When VNX Block Deduplication is enabled for VNX File, the Deduplication and Snapshot Savings reported for the Storage Pool is how much space has been saved using VNX Block Deduplication. The savings reported are - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 33
VNX Block Compression VNX Block Compression Overview In general, compression is a process that attempts to reduce the total space used by a dataset. VNX Block Compression works in 64 KB increments to reduce the storage footprint of a dataset and provide savings to the user. As accessing compressed - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 34
or Thick LUNs to Thin LUNs provides the benefits of recapturing and repurposing unused capacity and provides a "pay as you go" capacity consumption model. Users make this tradeoff when they do not have stringent IOPs and response time requirements. Compression takes this a step further, providing - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 35
without adding capacity to the pool, the user can manually override this safeguard and resume decompression. However, the compression user capacity is the same or larger than the compressed LUN's user capacity. Management An online LUN migration of Classic LUNs to Thin LUNs is performed when - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 36
Figure 31. Destination Pool Selection for Classic LUN Compression All currently eligible pools are listed in the Pool drop-down menu. Pools are only shown if they have enough free capacity to accommodate the user capacity of the Classic LUN. Capacity equal to the user capacity of the Classic LUN is - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 37
Figure 33. LUN properties after compression This savings represents the benefits of both moving the LUN from a Thick LUN to a Thin LUN and compressing the data. Space savings attained after compression is enabled on a Thick LUN in a pool entirely depends on the free space on the LUN and the - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 38
Figure 34. Pool Properties before compression Figure 35 shows the Pool Properties after compression of the 250 GB LUN. The Percent Full dropped from 9.53% to 8.058% and the free space increased from 11575.564 GB to 11763.932 GB. The capacity saved from compressing the LUN is returned to the pool for - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 39
To view the LUN compression states, see Unisphere Help for a list and more information about each state. The table located in the Help shows the compression state, a description of that state, and what LUN types this state is valid for. The Compressed LUNs Summary dialog box shown in Figure 36 - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 40
detailed synopsis of this command and others, please consult the VNX Command Line Interface Reference for Block documentation, located on EMC Online Support. Available Limits Table 2 Limits for the VNX Block Data Compression Feature VNX Model 5200/5400/5600 5800 Total Compressed LUNs Concurrent - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 41
enabled LUN cannot be used as a Clone Private LUN. A VNX Block Compression enabled LUN cannot be used as the Write Intent Log Not supported with VNX Block Compression. Block Compression with VNX File EMC strongly recommends that you create File Systems using thick, non-compressed LUNs. In VNX - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 42
using File Deduplication will vary as the savings entirely depend on the similarity of the files being stored on the file system. VNX File Compression Overview When enabled on a File System, VNX File Compression works to reduce the consumed size of files on disk. File size reduction is achieved by - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 43
Table 4 VNX File Deduplication and Compression File System/Data Mover Settings Setting Definition Default value Access Time Length of time in days that the file has not been accessed 15 days Case Sensitive Defines if case-sensitive (NFS environments) or case-insensitive (CIFS environments) - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 44
If the file is not in an excluded path and does not have an excluded file extension, the file is checked for the last time it was accessed and modified. These times are compared to the current settings, and if the file's last access and modified dates are older than the values currently set, the - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 45
Figure 38. VNX File Deduplication Detection Methods Choosing a Duplicate Detection Method As mentioned previously, File-level deduplication removes duplicate file contents using Single-Instance Storage. This means that duplicate file contents will be identified and single instanced as to reduce - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 46
replication software needs to be checked to see whether Deep compression is supported before it is used. CIFS Compression Enabled Table 7 shows an Definition CIFS Compression Enabled Enables CIFS compression Default Value On Management VNX File Deduplication and Compression is enabled on a file - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 47
Figure 39 below shows the File-Level Deduplication Enabled setting, which can be enabled at the time the File System is created. Figure 39. Deduplication Setting When Creating a File System If the user wishes to enable the Deduplication setting on an existing File System, the user first must open - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 48
location of this option. Figure 42 is an example of the Deduplication Settings window for the Data Mover. Figure 41. Deduplication Settings location Figure 42. The Data Mover Deduplication Settings Window File System level deduplication and compression settings In the File System window under the - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 49
Figure 43. The File System Properties Deduplication Settings Window Viewing Deduplication Statistics The user is able to view the results of the deduplication process on the file system data by using the file system's properties page. The deduplication savings is the total amount of space saved for - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 50
others, please consult the VNX Command Line Interface Reference for File documentation, located on EMC Online Support. Conclusion VNX storage systems provide powerful capacity efficiency features that can improve effective capacity utilizations. These capacity-optimization features are included with - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 51
savings entirely depend on the files being stored on the file system. References The following white papers are available on EMC Online Support: Virtual Provisioning For The VNX2 Series - Applied Technology EMC VNX2 Multicore FAST Cache EMC VNX2 FAST VP - A Detailed Review EMC VNX2 Unified - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 52
Appendix A: NTFS File System Conditioning with SDelete Many file systems do not efficiently reuse the space associated with deleted files. When files are deleted from NTFS file systems, the deleted files' data continues to be stored in the file system until it is overwritten by new data. When files - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 53
Appendix B: File Data Migration to VNX with Deduplication and Compression Example The following examples identify a customer wanting to migrate data from a Windows server to the VNX system. These real world scenarios show what happens when you migrate data and the importance of deduplication being - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 54
File System Deduplication Results Savings of Older NAS FsUtil 760GB 770,725 files can be archived 844,761 files can be deduplicated 69% of all files have gone 180+ days untouched 718GB combined size, or 75% of total size at 40% compression: 288GB potential savings After Migration ? ? ? to VNX - Dell VNX5600 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 55
Realized Savings - The level in which space savings is returned to Space Savings Mechanism - The software feature used to achieve space savings Supported LUN Type - The supported LUN types as the underlying storage Consideration Granularity Data Selection Realized Savings Space Savings Mechanism
White Paper
Abstract
This white paper discusses the capacity optimization
technologies delivered in the EMC
®
VNX
™
2 series of storage
platforms
.
Deduplication and compression capabilities for file
and block storage are delivered standard with the VNX
Operating Environment.
March 2016
EMC VNX2 Deduplication and Compression
VNX5200, VNX5400, VNX5600, VNX5800, VNX7600, & VNX8000
Maximizing effective capacity utilization