A study in VMAX & VNX auto-tiering

One of the major differences between a VMAX and VNX are the pooling & FAST-VP (auto-tiering SW) implementations. As more and more VNX customers are considering VMAX systems (thanks largely to the introduction of the VMAXe/VMAX10K price point) these differences are often a topic of conversation. There are some noticeable differences in the theory & operation and subsequently the real-world management which are worth understanding.

Data Movement Frequency & Data Movement Granularity:

The most obvious difference in the FAST-VP implementation between the two systems is how often data relocations can occur & the granularity of that data movement.

  • VMAX: data is collected continuously, analyzed continuously and can be moved continuously. The granularity of this data movement can be as small as 12 Symmetrix tracks or 768K
  • VNX: data is collected continuously, analyzed once an hour, and moved once per 24hrs. The way to think about auto-tiering on a VNX is that the system builds a “plan” based on a data collection window and then executes on that plan once every 24hrs during a specified relocation window. The granularity of data movement is 1GB

It’s important to take note of this major architectural difference between the two systems. The VNX the auto-tiering is designed as more of a slow moving interface and for this reason it should be considered mandatory that all VNX systems utilizing FAST-VP also have the appropriate amount of Fast Cache capacity. Fast Cache operates at 64K granularity and allows instant promotion of data. In the event there were some cold data living on NL-SAS that suddenly became hot, the VNX can immediately promote (or queue for promotion) it to Fast Cache to absorb that incoming workload spike while FAST-VP would come around on the next relocation window to decide if it makes sense to actually up-tier it permanently into the EFD pool drives. If that same scenario were to occur on the VMAX, the data would be promoted (or queued to the DA for promotion) almost instantly to the EFD tier. This is why the VMAX does not need the Fast Cache feature at a fundamental level (gobs of engine cache help too!)

Nerd knobs A.K.A. The “tunability” of FAST-VP across the two systems:

  • VMAX
    • Performance Time Window: when is the relevant time window to look at the data, usually set to 24/7/365 for continuous analysis
    • Data Movement Window: when can data be moved, usually set to 24/7/365 for continuous data movement
    • Workload Analysis Period: used to tune how to aggressively to decay “older” IO compared to “newer” IO for data promotion/demotion ranking purposes
    • Initial Analysis Period: on a newly created LUN, how long is data collected before doing the first movements
    • FAST-VP Relocation Rate: a number from 1 to 10 on how quickly FAST-VP should move data
  • VNX:
    • Data Relocation Rate: Low/Medium/High
    • Data Relocation Schedule: which days of the week & during what window should relocation run

It should be clear which system has a more “end-user tunable” FAST-VP system. To be fair, every customer does not tweak every single FAST-VP knob on the VMAX or any for that matter. The defaults or “best practices” (and how I hate that term!) actually work very well for the majority of the customers.

Taxonomy of storage pools & LUN Configuration Elements:

VNX:

The VNX has 4 main configuration elements as it pertains to this discussion:

  • the physical drives
  • the storage pool
  • the LUN
  • the tiering policy of the LUN
VNX Pool Abstractions (Annotated from pg 11 of VNX Virtual Provisioning White Paper)

VNX Pool Abstractions (Annotated from pg 11 of VNX Virtual Provisioning White Paper)

Based on the above, lets examine how to create some auto-tiered storage on the VNX.

Take the EFD/SAS/NL-SAS drives, create a pool:

VNX Create Pool

VNX Create Pool

Create a LUN in that pool:

VNX Create LUN

VNX Create LUN

Set the tiering policy on that LUN:

VNX Select Tiering Policy

VNX Select Tiering Policy

The tiering policies explained:
  • Start High Then Auto: place all the slices on the highest tier with available capacity (in this case EFD) and then examine the performance as time goes on and move the slices into the correct tiers
  • Highest: place the slices in the highest tiers possible and remain there, with the busiest slices in the highest tier
  • Auto: place the slices in the right tier based on performance thresholds, but initially slices are distributed throughout all 3 tiers (so some slices end up on EFD, some on SAS, and some on NL-SAS)
  • Lowest: place slices on the lowest available tier (in this case NL-SAS)
Assuming the first tiering policy (the recommended one for the majority of workloads), the LUN will have it’s slices start out on the highest tiers and over time auto-tier as appropriate.

VMAX:

The VMAX configuration elements are slightly different due to pooling architectural differences:

  • disk groups: collection of like type disks (I.E. 200GB EFD)
  • Virtual Pools: pooled storage capacity formed from disk groups
  • FAST-VP Tiers: association of a tier with the previously mentioned pooled capacity; multiple pools of like drive type & RAID protection type can be associated with a single FAST-VP Tier
  • FAST-VP Policies: auto-tiering policy specifying how much of each tier can be utilized I.E. 100/100/100 would tell the system that 100% of the EFD, 100% of the FC and 100% of the SATA tier can be utilized for a given LUN/Storage Group
  • storage groups: collection of host LUN
VMAX Pool Abstractions

VMAX Pool Abstractions

On the VMAX each type of drive is associated with a disk group:

VMAX Disk Groups

VMAX Disk Groups

Then virtual pools can be created from each of the drive types. A RAID protection and capacity are specified for each pool (can be full, or a subset of the entire system capacity based on the disk groups):

VMAX Pools

VMAX Pools

Next FAST-VP Tiers need to be created and associated with the Virtual Pools. Typical would be an EFD Tier, a FC Tier, and a SATA Tier. The example below shows the creation of an “EFD” FAST-VP Tier.

Creating a VMAX FAST-VP Tier

Creating a VMAX FAST-VP Tier

VMAX FAST-VP Tiers

VMAX FAST-VP Tiers

All 3 tiers have been created above.

Next comes the FAST Policy. This determines the % of each tier (by capacity) that a LUN can occupy. 100/100/100 is the ideal policy as it basically tells the system “I’m not placing any restrictions on the tier percentages, you decide the best place to land the data”. This gives the system full control and if the VNX had a FAST-VP Policy parameter, this would be the setting (100/100/100):

Creating a FAST Policy

Creating a FAST Policy

idealP FAST Policy Created

idealP FAST Policy Created

Alternatively, different policies can be created such as 20/30/50 which tells the system at most 20% EFD by capacity and 30% FC by capacity can be utilized and the rest needs to live on the SATA tier. So again, the FAST Policy on the VMAX allows plenty of “nerd knob” tweaking if desired. Other uses for this would be in multi-tenant or storage as a service model where internal/external customers are paying different $/GB rates based on expected SLAs.

Next a storage group and host devices (LUN) must be created. This is as expected, but one item that needs to be specified is the initial pool binding. Meaning, which virtual pool to associate the LUN with initially. The best practice is to choose the middle / FC tier for this. Note that if  thin provisioning is used, no space is actually occupied in pool until host writes are sent. Additionally, there is a setting in VMAX 5876 code which allows new writes to land on a tier that FAST-VP decides is best based on the data collected on host IO activity. With this setting, FAST-VP may decide to land the new write on SATA even though the initial binding was on FC if it deems appropriate. This avoids tracks landing on FC first, only to be moved down to SATA later (or up to EFD). This is a system wide setting called “allocate by FAST policy” and the recommendation is to enable it unless there is a good reason not to.

VMAX Create Storage Group

VMAX Create Storage Group

VMAX Create Storage Group

VMAX Create Storage Group

Next the storage group has to be associated with a FAST Policy:

VMAX Associate a FAST Policy

VMAX Associate a FAST Policy

… and now the LUN is auto-tiered via the pools created utilizing the FAST Policy specified (100/100/100 in this case).

The FAST Policy can be changed anytime on the fly. For example it could be changed from 100/100/100 to 20/30/50 or any combination based on business needs. This gives a lot of flexibility in the management of the performance & capacity of the array.

To summarize the data movement process between a VMAX and VNX as it pertains to auto-tiering:

-VMAX: TDEV (LUN) is bound to a pool/tier (best practice FC unless low workload); after the Initial Analysis Period performance metrics are analyzed; extents are marked for promotion / demotion; data movements queued up on the DA (disk adapters); TDEV remains bound to the pool it was originally bound (for statistics purposes) regardless of where the tracks live;new host writes behavior depends on the “allocate by FAST Policy” setting.

-VNX: LUN created in a Pool; initial allocation determined by Tiering Policy of LUN; data collected immediately, analyzed every hr, and if necessarily slices will be moved during next relocation window once every 24hrs

Other notable behavioral differences:

  • on a VNX, extents are only moved down to lower tiers if space is needed on higher tiers to accommodate up-movement. On the VMAX, extents will be proactively moved down if the system deems them of lesser performance regardless of available capacity in higher tiers.
  • during replication the VNX on the target side has no information on FAST-VP movements that have occurred. The VMAX does have a feature called “FAST-VP RDF Coordination” which when enabled at the storage group level will allow the source VMAX to communicate with the target VMAX on FAST-VP movements, and thus the R2 volumes on the target side will have data located on the appropriate tier as per the R1 workload. Note: This is for SRDF ONLY, and does not work in Recoverpoint Environments (as of today)
  • in environments where block side compression is being utilized, the VMAX can automatically compress inactive data under FAST-VP management with a toggle of between 40-400 days of inactivity. The compression on the VNX is more manual in nature with a simple enable/disable mechanism.

Summary:

Its important to understand the differences in pooling & FAST-VP between the two systems. The VMAX offers -much- more flexibility in how the pools & FAST-VP can be configured, however not all customers require it. The reason for many of the differences is due to how much more global memory & processing power a VMAX has compared to the VNX. This is especially true of the VMAX40K and the new VMAX10K “989” engines.

Comments/questions welcome.



Categories: EMC, storage, VMAX, VNX

10 replies

  1. Reblogged this on DanWolkenstein and commented:
    This is an awesome in-depth explanation of two leading storage platforms in the industry!

  2. Reblogged this on Phil's Splice and commented:
    Very informative comparison of platforms!

  3. Excellent writeup! This is one of the best writeups i have come across describing very well how the VNX and the VMAX works.
    Excellent job Vijay! Keep up the good job!

  4. Hi Vijay!
    Could you tell me what will be if I set “Data Relocation Duration” to 24 hours in the VNX?
    Will I get continuous relocation?

  5. Nicely done!!…

  6. On what parameters does the FAST algorithm decide to move data. for example, if the latency reaches a threshold, say 8ms? (if Yes, how do i configure it). What happens if 2 storage group with the same policy, is in contention of EFD?

  7. I have a quick Question… In VNX if I start with ‘Start high then auto tier’ and later change it to ‘Lowest tier’ will all the data eventually come down to lowest tier? or the data at the highest tier will stay there if it is being actively used?

  8. Quick question. Irrespective of what technology used, you create a tier of storage layers with different performance potential. This can be expressed as a ratio between the tiers to provide a figure describing the performance potential of the system,

    Now if a customer comes along with an additional storage requirement, how can additional storage be added to the system whilst retaining the ratio between tiers so existing customers are not impacted by changed performance by the addition of storage for the new customer ?

  9. I have my tier 1 EFD 1-TB and it is 90% full and may increase getting more data. What happens if it reaches 100% and more data has to be moved to this? will this cause issues?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: