Dell VNX8000 VNX2 Deduplication and Compression - Maximizing effective capacit
Dell VNX8000 Manual
View all Dell VNX8000 manuals
Add to My Manuals
Save this manual to your list of manuals |
Dell VNX8000 manual content summary:
- Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 1
White Paper EMC VNX2 Deduplication and Compression VNX5200, VNX5400, VNX5600, VNX5800, VNX7600, & VNX8000 Maximizing effective capacity utilization Abstract This white paper discusses the capacity optimization technologies delivered in the EMC® VNX™2 series of storage platforms. Deduplication and - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 2
Copyright © 2016 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate of its publication date. The information is subject to change without notice. The information in this publication is provided "as is." EMC Corporation makes no representations or - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 3
Servers 22 Common Data Environment 22 Performance and Capacity Savings 22 VNX Block Deduplication for VNX File 23 Requirements and Supported Configurations 23 Management ...25 Considerations ...32 VNX Block Compression 33 VNX Block Compression Overview 33 The VNX Block Compression Process - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 4
Data Mover level deduplication and compression settings 47 File System level deduplication and compression settings 48 Viewing Deduplication Statistics 49 VNX File Deduplication and Compression commands 50 Conclusion ...50 References ...51 Appendix A: NTFS File System Conditioning with SDelete - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 5
built to handle the I/O demands of a large number of Flash drives. Performance-optimization features such as Multicore FAST Cache and Fully Automated Storage the VNX2 Series (VNX5200, VNX5400, VNX5600, VNX5800, VNX7600, and VNX8000). As part of EMC's effort to improve its products, EMC periodically - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 6
VNX Block Deduplication VNX Block Deduplication is a software feature included with the VNX2 series. In general, deduplication is the process of identifying duplicate data contained within a set of block storage objects and consolidating it such that only one actual copy of the data is used by many - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 7
Figure 2. VNX Block Deduplication example Figure 3 shows the process of deduplicating data on LUNs 1, 2, and 3. The first instance of each 8 KB chunk of data is kept in the pool, while the duplicates are deleted. Each LUN containing the chunks of data that were deduplicated will point to the copies - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 8
In the above example, the deduplication process removes the duplicates of the original of 8 KB chunks. Consider a storage array which stores multiple copies of the same data. In this scenario, the array needlessly copies much of the data repeatedly. Deduplication consolidates most of the 8KB - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 9
enabled, but the Allocation Owner will change to match the owner of the deduplication container. A LUN's Allocation Owner can be viewed on the General tab within the LUN Properties window in Unisphere. This is also where you determine which SP owns the pool's deduplication container by viewing the - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 10
container cannot be changed without disabling deduplication on all LUNs within a pool. Figure 5. Deduplication Container Balance Example VNX Block Deduplication runs in the background on the contents of a deduplication container post process, 12 hours after the previous deduplication run completed - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 11
slice may be freed back to the pool for use when the slice no longer contains any data. If a deduplication process has run on a pool for 4 hours straight, the run is paused and the system checks for deduplication processes waiting to run on that SP. If a deduplication process is waiting to run, it - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 12
Another improvement in the VNX OE for Block version 05.33.009.5.155 release has been made to the deduplication process and highly referenced blocks. When the deduplication process runs, there is a potential for a single 8 KB block to be referenced hundreds, or even thousands of times. This - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 13
, or the array is running an earlier version of VNX OE for Block. If running on a code previous to 05.33.000.5.072, you must manually set the Default Owner of the LUN to the SP that owns the deduplication container within the pool. Figure 7: Create LUN Advanced tab EMC VNX2 - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 14
If the user changes the Default Owner to the peer SP in VNX OE for Block version 05.33.000.5.072 and later, the message shown in Figure 8 is displayed. By changing the Default Owner, the user is choosing to avoid the optimal ownership model for the LUN. As the message indicates, changing the owner - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 15
. LUN properties Deduplication tab for a VNX File LUN When deduplication is enabled on any LUN, the system will automatically store the original LUN type, allocation SP, default SP, and FAST VP Policy. If deduplication is disabled on a LUN, that LUN is automatically migrated out of the deduplication - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 16
of SPB. After enabling deduplication on an existing LUN, the user should verify that the Default Owner and Current Owner match the Allocation Owner of the LUN. If the Default Owner does not match the Allocation Owner, the Default Owner should be changed by using the Properties page of the LUN in - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 17
Shortly after deduplication is enabled on a LUN, the LUN state will show as enabling in the Deduplication tab in the LUN properties window. While the LUN State shows as enabling, the LUN is migrating into the deduplication container within the pool. At any time, the user can view deduplication - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 18
The Deduplication tab (Figure 14) is where you set the FAST VP Tiering Policy for all of the deduplicated LUNs in this pool. The data in the deduplication container is treated as a whole; thus the Tiering Policy applies to all deduplicated data on the pool. The same options for the Tiering Policy on - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 19
on each of these commands and others, please consult the VNX Command Line Interface Reference for Block documentation, located on EMC Online Support. Block Deduplication with Multicore FAST Cache, FAST VP, and Multicore Cache The basic purpose of VNX Block Deduplication is to eliminate data - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 20
After duplicate data has been removed within a deduplication container, FAST VP operations become more efficient as there is less data to track and promote. Also, with a consolidated storage footprint, FAST VP can better use space on the pool. Consider multiple LUNs on the same pool that have been - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 21
LUN that is already in the midst of migrating). Similarly, a LUN that is currently migrating cannot have its deduplication-enabled state changed. VNX Snapshots Fully supported. Cannot take a snapshot while the LUN is being migrated in or out of the pool (when the LUN is in "enabling" or "disabling - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 22
will thus be common with other base images of that type. This environment of common data is ideal for Block drives increase and larger portions of the workload is served out of Flash, the system utilizes more CPU resources to maximize that potential. Block Deduplication is a data service - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 23
File resource. If these requirements are not met, a warning appears and deduplication is not enabled. Enabling VNX Block Deduplication is only supported on Mapped Storage Pools or Meta Volume based File Systems and Checkpoint Storage. The underlying block storage for these configurations must be - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 24
which deduplication is enabled, only expanding the resource with whole disk volumes is supported. When the extension is started, the new LUN will inherit the deduplication the system if the need arises. Split log is the default log type in VNX OE for File version 8.1.6.96 and later, and is strongly - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 25
Management VNX Block Deduplication is typically enabled on Block Storage Pool LUNs directly via Unisphere or NaviSecCLI. In VNX OE for Block version 05.33.008.5.119 and VNX OE for File 8.1.8.119, user controls for Block Deduplication have been disabled for LUNs provided to VNX File. Controlling of - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 26
Figure 18. Mapped Storage Pool Properties Window. After enabling or disabling VNX Block Deduplication, a migration into or out of the deduplication container begins. To view which LUNs are migrating, view the LUNs within the Block Storage Pool, as shown in Figure 19. If the Deduplication column is - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 27
Deduplicated LUN SP Owner within the Deduplication tab of the Storage Pool Properties window. Once the optimal owner is determined, the user must manually change the Default Owner, shown in Figure 12, on each of the affected LUNs and issue nas_storage -check -all and nas_storage - failback commands - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 28
Figure 21. Could not enabled VNX Block Deduplication error. Figure 22 below shows an error seen while deduplication is enabling or disabling on the underlying LUNs. The error informs the user that not all LUNs have a deduplication state which matches the setting on VNX File. Listed in this error are - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 29
example was previously creating using two disk volumes. Using the Control Station CLI command nas_fs -create with the -fixed_block_dedup option allows the user to specify if block deduplication is enabled or disabled when creating a File System on a Meta Volume. Figure 24. File System on Meta Volume - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 30
. When VNX Block Deduplication is enabled on the Checkpoint Storage, Auto Extend is disabled. To extend the Checkpoint Storage the user must manually extend it using a Meta Volume utilizing whole disk volumes. If the Checkpoint Storage is created from a Storage Pool, FixedBlock Deduplication Enabled - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 31
Figure 27. shows the Enter File System Info step within the New File System Wizard. When using the wizard to create a File System on a Mapped Storage Pool, the user will be presented with the state of Fixed-Block Deduplication for the Mapped Storage Pool. As VNX Block Deduplication is enabled at the - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 32
strongly recommends that you create File Systems using thick LUNs with Block Deduplication disabled. In VNX OE for Block Release 33, the default LUN type for a new LUN is thin; this can be accomplished by unchecking the 'Thin' checkbox when creating the LUN. Block Deduplication will be disabled by - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 33
completed, compression is not suggested to be used on active datasets. The VNX Block Compression Process VNX Block Compression can be enabled on any LUN type on the system. Once compression is enabled on a pool-based LUN, an initial compression process is run and the LUN becomes Thin. If Thin LUN - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 34
Compression Rate Moving datasets from Classic LUNs or Thick LUNs to Thin LUNs provides the benefits of recapturing and repurposing unused capacity and provides a "pay as you go" capacity consumption model. Users make this tradeoff when they do not have stringent IOPs and response time requirements. - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 35
decompression without adding capacity to the pool, the user can manually override this safeguard and resume decompression. However, the compression to decompression. This option might be chosen when a change in LUN type may be warranted and/or different physical spindles may be used. Users - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 36
Figure 31. Destination Pool Selection for Classic LUN Compression All currently eligible pools are listed in the Pool drop-down menu. Pools are only shown if they have enough free capacity to accommodate the user capacity of the Classic LUN. Capacity equal to the user capacity of the Classic LUN is - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 37
Figure 33. LUN properties after compression This savings represents the benefits of both moving the LUN from a Thick LUN to a Thin LUN and compressing the data. Space savings attained after compression is enabled on a Thick LUN in a pool entirely depends on the free space on the LUN and the - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 38
Figure 34. Pool Properties before compression Figure 35 shows the Pool Properties after compression of the 250 GB LUN. The Percent Full dropped from 9.53% to 8.058% and the free space increased from 11575.564 GB to 11763.932 GB. The capacity saved from compressing the LUN is returned to the pool for - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 39
list and more information about each state. The table located in the Help shows the compression state, a description of that state, and what LUN types this state is valid for. The Compressed LUNs Summary dialog box shown in Figure 36 provides a consolidated view of block compression activity for all - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 40
detailed synopsis of this command and others, please consult the VNX Command Line Interface Reference for Block documentation, located on EMC Online Support. Available Limits Table 2 Limits for the VNX Block Data Compression Feature VNX Model 5200/5400/5600 5800 Total Compressed LUNs Concurrent - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 41
Compression enabled LUN cannot be used as the Write Intent Log Not supported with VNX Block Compression. Block Compression with VNX File EMC strongly recommends non-compressed LUNs. In VNX OE for Block Release 33, the default LUN type for a new LUN is thin so this can be accomplished by unchecking - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 42
using File Deduplication will vary as the savings entirely depend on the similarity of the files being stored on the file system. VNX File Compression Overview When enabled on a File System, VNX File Compression works to reduce the consumed size of files on disk. File size reduction is achieved by - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 43
Table 4 VNX File Deduplication and Compression File System/Data Mover Settings Setting Definition Default value Access Time Length of time in days that the file has not been accessed 15 days Case Sensitive Defines if case-sensitive (NFS environments) or case-insensitive (CIFS environments) - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 44
If the file is not in an excluded path and does not have an excluded file extension, the file is checked for the last time it was accessed and modified. These times are compared to the current settings, and if the file's last access and modified dates are older than the values currently set, the - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 45
Figure 38. VNX File Deduplication Detection Methods Choosing a Duplicate Detection Method As mentioned previously, File-level deduplication removes duplicate file contents using Single-Instance Storage. This means that duplicate file contents will be identified and single instanced as to reduce - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 46
was compressed using the Deep method is an I/O error. The replication software needs to be checked to see whether Deep compression is supported before it is used. CIFS Compression Enabled Table 7 shows an option for controlling whether compression is enabled on CIFS. By default, CIFS compression - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 47
Figure 39 below shows the File-Level Deduplication Enabled setting, which can be enabled at the time the File System is created. Figure 39. Deduplication Setting When Creating a File System If the user wishes to enable the Deduplication setting on an existing File System, the user first must open - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 48
location of this option. Figure 42 is an example of the Deduplication Settings window for the Data Mover. Figure 41. Deduplication Settings location Figure 42. The Data Mover Deduplication Settings Window File System level deduplication and compression settings In the File System window under the - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 49
Figure 43. The File System Properties Deduplication Settings Window Viewing Deduplication Statistics The user is able to view the results of the deduplication process on the file system data by using the file system's properties page. The deduplication savings is the total amount of space saved for - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 50
Interface Reference for File documentation, located on EMC Online Support. Conclusion VNX storage systems provide powerful capacity efficiency and block storage offer complementary capacity efficiency opportunities for all data types in the primary storage systems. VNX Block Deduplication is another - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 51
savings entirely depend on the files being stored on the file system. References The following white papers are available on EMC Online Support: Virtual Provisioning For The VNX2 Series - Applied Technology EMC VNX2 Multicore FAST Cache EMC VNX2 FAST VP - A Detailed Review EMC VNX2 Unified - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 52
Appendix A: NTFS File System Conditioning with SDelete Many file systems do not efficiently reuse the space associated with deleted files. When files are deleted from NTFS file systems, the deleted files' data continues to be stored in the file system until it is overwritten by new data. When files - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 53
Appendix B: File Data Migration to VNX with Deduplication and Compression Example The following examples identify a customer wanting to migrate data from a Windows server to the VNX system. These real world scenarios show what happens when you migrate data and the importance of deduplication being - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 54
File System Deduplication Results Savings of Older NAS FsUtil 760GB 770,725 files can be archived 844,761 files can be deduplicated 69% of all files have gone 180+ days untouched 718GB combined size, or 75% of total size at 40% compression: 288GB potential savings After Migration ? ? ? to VNX - Dell VNX8000 | VNX2 Deduplication and Compression - Maximizing effective capacit - Page 55
Realized Savings - The level in which space savings is returned to Space Savings Mechanism - The software feature used to achieve space savings Supported LUN Type - The supported LUN types as the underlying storage Consideration Granularity Data Selection Realized Savings Space Savings Mechanism
White Paper
Abstract
This white paper discusses the capacity optimization
technologies delivered in the EMC
®
VNX
™
2 series of storage
platforms
.
Deduplication and compression capabilities for file
and block storage are delivered standard with the VNX
Operating Environment.
March 2016
EMC VNX2 Deduplication and Compression
VNX5200, VNX5400, VNX5600, VNX5800, VNX7600, & VNX8000
Maximizing effective capacity utilization