Building on the Bare Minimum

    When the need arises to upgrade from a bare minimum storage configuration or to start off with a more scalable storage configuration for VMware, Sun Storage arrays include all of the configuration combinations and choices needed. Virtualized Data Centers rely on on-demand highly scalable storage infrastructure in order to perform optimally. The need for more storage can increase exponentially over time and the storage infrastructure including storage arrays and the SAN need to be highly flexible in terms of scalability. The table below illustrates the scalability of Sun Storage arrays:

For latest and detailed Sun Storage architectural information, please refer to www.sun.com/storage.

Given the extensive possibilities on scalability for multiple architectural scenarios, Sun Storage arrays provide a very robust set of products and capabilities specifically for virtualization technologies such as VMware. In order to achieve optimal storage availability, performance and other architectural considerations, simple additions to the bare minimum designs may be sufficient.

Segregating Different Data Types

 For an optimal VMware infrastructure, segregating different types of data is essential. VMware makes use of storage for a number of reasons such as:

- Virtual Machine home directory which includes the VM Operating System, virtual memory and other essential objects
- Critical VM home directories should be segregated from Non-Critical VMs
   = Software ISO images for easy sharing and access of OS and application install files
- User and Application Data which may include critical and non-critical data such as User home directories and Employee Pay Roll

The need for Data Type Segregation arises due to the unique data access requirements for each. VM home directories have unique data access needs because the data includes the operating system and virtual memory of the VM which could lead to slow VM performance if placed along with random data type such as an Oracle OLTP  high performance random I/O database. Here are recommendations and strategies to address the issue of data segregation. :

1) Segregate Virtual Machine Home Directories based on the type of application and I/O. For example:
    - Test VMs and Production VMs
    - Database VMs which have higher memory requirements and may have higher swap file usage should not be on the same array volumes
2) Random I/O and Sequential I/O data should reside in different RAID Groups or Virtual Disks.
3) Use Luns or Volumes from different RAID Groups or Virtual Disks for different data types.
    - Luns for Virtual Machine Home Directories should not share the same RAID Group or Virtual Disk as an Oracle OLTP Database Lun
    - Non-Critical data such as ISOs and Non-Critical VM home directories may share the same RAID Group.
4) Use Low-Cost Modular Sun Storage arrays such as the Sun Storage 2510 for ISOs and non-critical VMs.
- VMs can boot from iSCSI targets on the ESX Server
    - Sun Storage 2510 can be shared across multiple ESX servers in order to take advantage of all VMware features
5) Mid-range modular for mid-range performance
    - Based on user requirements
6) High performance I/O intensive data such as Oracle OLTP, Data Warehousing should reside on high-end 6580/6780 and 9000
7) Use of Storage Domains on Sun Storage 2500 and 6000 series products is highly recommended in order to isolate ESX servers from other servers. Using Storage Domains allows creation of virtual partitions on the storage array and enables lun mapping per host initiator, host or host group. The following table shows Storage Domain feature scalability of the arrays.For more details on Storage Domains, please refer to Sun Storage Common Array Manager Installation Guide.

SAN switch zoning should also be used to segregate traffic for each initiator or HBA along with lun mapping and storage domains. Zoning is primarily used to isolate Initiator – to – Target traffic and disallows SAN error propagation across zones, the most important one being Registered State Change Notification events. An RSCN is a required notification sent out to all devices of a particular SAN, switch or a particular zone or set of zones every time due to a change in the SAN, causing the I/O traffic to halt momentarily. The change could be a host reboot, a cable plug/un-plug, a hardware error, a new storage device  and similar events. This I/O traffic halt, although brief, has been known to cause I/O exchange failures, path failures and in some very extreme situations data corruption See Figure 2 below. With zoning implemented, effects of RSCN are minimized to the specific zone which experienced the change event. Best practice for SAN zoning is to use one initiator (Host Bus Adapter) per zone and if possible, one target (Array port) per zone. Zoning also helps isolate Non-ESX server traffic from ESX server traffic.

In the figure above, there are three zones illustrated. Zone A and Zone C both have one initiator and one target, while Zone B has two initiators and targets. Both switches in the SAN are connected via an Inter-Switch Link. If Host X rebooted and it’s HBA in Zone B logs out of the SAN, an RSCN will be sent to Host Y’s initiator in Zone B and cause all I/O going to that initiator to halt momentarily and recover within seconds. However, another RSCN will be sent out to Host Y’s initiator in Zone B when Host X’s HBA logs back in to the SAN and cause another momentary halt in I\O. Initiators in Zone A and Zone C are protected from these events because there are no other initiators in these zones. Most latest SAN switches provide RSCN suppression methods, however, suppressing RSCNs is not recommended since RSCNs are the primary way for initiators to determine an event has occurred and to act on the specified event such as lost of access to targets. It is important to follow established SAN best practices such as single initiator zones in order to avoid situations described and others not listed.

Enhancing Availability

    VMware simply requires shared storage to enable all advanced features addressing Availability at the Virtual Machine level as long as there are more than one ESX servers sharing the same volume in a storage array. There is, however, a need to address Data Availability at the storage array level as well. Although Sun Storage arrays have built-in redundancy for data access, cache mirroring across controllers and power, and the likely hood of a complete failure is very small, it is recommended, if possible, to spread the data across multiple disk trays via mirror and/or striping on the low cost arrays specifically. Sun Storage 2500 series, 6000 series and StorageTek 9900V series offer modular add-on capabilities to attach disk trays/chassis/frame to an existing set of array controllers with no downtime in most cases. Striping data across multiple trays using RAID 5,6 or 1+0 enhances data availability by avoiding loss of access due to complete disk tray failure. Consider three virtual-disks, 4 disk RAID-5, a 5 disk RAID-6 and a 4 disk RAID-1+0, stripped/mirrored across a Sun Storage 2540 with four trays as shown in the figure below:

In this example, all virtual-disks will continue to function properly even with the loss of one entire disk tray. The RAID-6 virtual-disk will continue to to function even with the loss of two complete disk trays as long as the two trays did not include the bottom one. Similarly, The RAID-1+0 virtual-disk can sustain a combination of entire disk tray failures. This is just an example of how striping or mirroring across disk trays enhances availability and eliminates array single points of failures. For VMware environments where multiple servers are consolidated onto just a few physical machines and the data, including operating systems and virtual machine home directories, resident on shared storage, it is strongly recommended that RAID configurations similar to the example above be used to stripe and/or mirror data across multiple disk trays to avoid storage array single points of failures.

Enhancing Performance

    Storage I/O performance for VMware  can be enhanced using different strategies. Since VMware VI3  does not officially support Load-Balancing for Storage Multi-Pathing and only one path for each Lun is active on the ESX server at any given time regardless of the total number of paths available for use. Some important strategies include:

1) Lun Mapping should be balanced across each available Target Host Ports.
     - Spreading luns across all available target host ports is a best practice. This avoids overloading one target host port while other available ports are idle.
2) Avoid mapping more heavily used Luns to the same Target Host Ports unless the luns are mapped to a group of ports to further enhance availability.
- Preferred Owning Controller on the array and/or Preferred Path on VMware should be configured for each heavily used Lun mapped to multiple target ports such that one heavily used lun will use a different preferred target port over the other. It is recommended to use VMware to specify Preferred Path for Active/Active arrays such as the Sun StorageTek 9900 series and to set Preferred Owning Controller at the array for Active/Passive or Asymmetric arrays such as the Sun Storage 2500 and 6000 series arrays.

- In Figure-4 above, four Fibre Channel Host Bus Adapter ports on a Sun fire X4600-M2 are mapped to a Sun Storage 6780 array with one CSM200 disk tray through a fibre channel SAN. All four volumes are mapped to four Target Host Ports while one controller is the Preferred Owning Controller for each of the volumes as indicated above. Sun Common Array Manager can be used to configure Preferred Owning Controller for Sun Storage 2500 and 6000 series asymmetric arrays as show in Figure-5 below.

- VMware Preferred Path selection can be used for Active/Active storage arrays where all controllers on the subsystem present all paths to the ESX server as active paths for I/O and the server uses the specified path to be the Preferred Path when available. This can be done using VMware vCenter client as shown in Figure-6 below.
- It is not recommended to use both VMware and the array to specify preferred path. This can lead to Path Thrashing in case of a path failure or similar event. VMware will try to use any next available path while the array may want to force the Lun to use a different target port on the preferred controller if the lun was mapped to more than one target ports on the preferred controller. It is also not recommended to apply VMware Fixed Path Policy to an Asymmetric or Active/Passive controller array as it can lead to performance related and path thrashing problems as well. For further details on Path Thrashing, refer to VMware’s SAN Configuration Guide.

3) Striping and mirroring across disk trays also enhances performance since I/O is separated across different back-end loops instead of the same loop being used to access disks on the same disk tray
4) RDM luns in VMware may offer slight performance improvement over VMFS volumes. If performance is a priority over some VMware features such as Storage VMotion, RDM can be the disk choice in order to slightly improve storage I/O performance. RDM volumes may be required for clustering such as Microsoft Cluster Service (MSCS) or data services on the storage arrays. More information on VMware RDM luns can be found in VMware’s SAN Configuration Guide.

Tiered Storage Architecture

    Sun Storage Tiered Storage Architecture offer a multitude of possibilities, specifically for Virtualization technologies such as VMware. Tiered Storage can simplify Storage Infrastructure management by consolidating many storage sub-systems behind one which allows management and provisioning of the entire storage infrastructure from one storage array. Some uses of Tiered Storage are:
1) Consolidating Storage Sprawl behind one Storage Sub-System
    - Allows single point storage infrastructure management and administration
    - Allows single point lun storage provisioning
2) Protecting investment by attaching older storage sub-systems or disk trays behind newer and improved sub-system controllers. Luns resident on slower, older sub-systems or disk trays can be used for:
- Non-critical data such as ISO images, Test/Development Virtual machine home directories
    - Data Archiving and backup using in-system replication of production Luns to tiered sub-system and then provisioning the replicated luns to backup servers
    - Data not requiring high performance I/O
    - Data migration
    - Virtual Machines on older arrays may not have to be migrated to newer sub-systems. Tiered Storage can be used to provision the same luns to the ESX servers. A simple rescan of the luns from the ESX servers is sufficient to recognize the volumes and virtual machines on the luns. See Figures-7 below.

Sun Storage 6000 Series Tiered Storage Capabilities

    Sun Storage 6000 series series arrays offer unique Tiered Storage Architecture capabilities. A Sun Storage 6540 can be converted into the next generation Sun Storage 6780/6580 simply by replacing the 6450 array controller tray with a 6580/6780 controller tray. An existing 6140 can be moved under a 6580/6780 control by converting the 6140 controller tray into a Sun Storage CSM200 disk tray and then attaching to the 6780/6580 controller tray along with any existing CSM200 disk trays in the 6580/6780. This procedure allows migration of existing data and storage arrays, with data intact protection, under newer next generation 6580/6780 arrays which offer enhanced performance, scalability and reliability. See Tables 1 and 3 above. Users are able to re-use and protect capitol investment as a result of this capability.

NOTE: This procedure requires Sun Support Services on-site assistance. Failure to do so may result in complete data loss.

See Figure-8 below for an illustration of how a Sun Storage 6140 and 6540 is moved under Next-Generation a 6780 control.

Sun StorageTek 9900V Series Tiered Storage Capabilities

With Sun StorageTek 9900V series arrays’ External Storage support, a complete Tiered Storage architecture can be created. Sun StorageTek 9900V offer enterprise class Tier-1 storage scalability, availability and performance (See Tables 1 and 3). Tier-2 and Tier-3 storage arrays can be virtualized behind Sun StorageTek 9900V arrays as external arrays. Luns or volumes from external arrays are provisioned through the front-end Sun StorageTek 9900V to the SAN for access by servers. A wide range of Sun Storage and Non-Sun Storage arrays are supported as External Storage for the Sun StorageTek 9900V. Additional capacity licenses may be needed to enable External Storage support and to take advantage of premium features such as Shadow Image (In-system replication), TrueCopy (Remote Replication), Dynamic Provisioning (Thin Provisioning) and others. Table-5 provides information on total external storage capacity currently supported with Sun StorageTek 9900V arrays.

Figure-9 is an illustration of a Sun StorageTek 9990V virtualizing an EMC Clariion and a Sun StorageTek 9970 array using the External Storage Feature for a Tiered Storage Architecture spreading older and non-critical data to older arrays while keeping critical Guest OS data, non-virtualized server data and high-performance data on the high performance newer array.

More details on Sun StorageTek 9900V External Storage Feature can be found on www.sun.com/storage

References

Sun Storage 6580 and 6780 Release Notes
- http://dlc.sun.com/pdf/820-5776-11/820-5776-11.pdf
Sun Storage 6580 and 6780 Hardware Installation Guide
- http://dlc.sun.com/pdf/820-5773-10/820-5773-10.pdf
Sun Storage 2500 Series Array Hardware Installation Guide
- http://dlc.sun.com/pdf/820-0015-12/820-0015-12.pdf
Sun Storage 6140 Array Hardware Installation Guide
- http://dlc.sun.com/pdf/819-7497-11/819-7497-11.pdf
Sun Storage Common Array Manager Software Installation Guide
- http://dlc.sun.com/pdf/820-5747-10/820-5747-10.pdf
Using RAID 6 for Increased Reliability Sun Blueprint
- http://wikis.sun.com/display/BluePrints/Using+RAID+6+for+Increased+Reliability+and+Performance
VMware Storage VMotion Best Practices for Sun Storage 2500 and 6000 Series Arrays
- http://www.sun.com/storage/white-papers/wmware-storage-VMotion.pdf
VMware VI3 and Virtual Infrastructure Basic System Administration Guide
- http://www.VMware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_admin_guide.pdf
VMware Storage/SAN Compatibility Guide
- http://www.VMware.com/resources/compatibility/pdf/vi_san_guide.pdf
VMware Remote Command-Line Interface Installation and Reference Guide
- http://www.VMware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_rcli.pdf
VMware vCenter Site Recovery Manager Compatibility Guidelines
- http://www.VMware.com/pdf/srm_storage_partners.pdf
VMware Fibre Channel SAN Configuration Guide
- http://www.VMware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_san_cfg.pdf

Source/Kaynak : http://blogs.sun.com/saidsyedblog/entry/sun_storage_sizing_and_design2