Best Practices For Designing, Deploying, And Administering .

IV. ESSENTIALSDomain IDA byte-wide field in the three byte Fibre Channel address that uniquely identifies a switch in afabric. The three fields in a FCID are domain, area, and port. A distinct Domain ID isrequested from the principal switch. The principal switch allocates one Domain ID to eachswitch in the fabric. A user may be able to set a Preferred ID which can be requested of thePrincipal switch, or set an Insistent Domain ID. If two switches insist on the same DID one orboth switches will segment from the fabric.Director SwitchesAn enterprise-class Fibre Channel switch, such as the Connectrix ED-140M, MDS 9509, orED-48000B. Directors deliver high availability, failure ride-through, and repair under power toinsure maximum uptime for business-critical applications. Major assemblies, such as powersupplies, fan modules, switch controller cards, switching elements, and port modules, are allhot-swappable. The term director may also refer to a board-level module in the Symmetrix that provides the interface between host channels (through an associated adapter module inthe Symmetrix) and Symmetrix disk devices. Interswitch link (ISL) a physical E Portconnection between any two switches in a Fibre Channel fabric. An ISL forms a hop in afabric.HBAA bus card in a host system that allows the host system to connect to the storage system.Typically, the HBA communicates with the host over a PCI or PCI Express bus and has asingle Fibre Channel link to the fabric. The HBA contains an embedded microprocessor withon-board firmware, one or more ASICs, and a Small Form Factor Pluggable module (SFP) toconnect to the Fibre Channel link.FabricOne or more switching devices that interconnect Fibre Channel N Ports, and route FibreChannel frames based on destination IDs in the frame headers. A fabric provides discovery,path provisioning, and state change management services for a Fibre Channel environment.LUNIn computer storage, a logical unit number (LUN) is simply the number assigned to a logicalunit. A logical unit is a SCSI protocol entity, the only one which may be addressed by theactual input/output (I/O) operations. Each SCSI target provides one or more logical units, anddoes not perform I/O as itself, but only on behalf of a specific logical unit.2010 EMC Proven Professional Knowledge Sharing11

NASNetwork-attached storage (NAS) is file-level computer data storage connected to a computernetwork providing data access to heterogeneous network clients. A NAS unit is essentially aself-contained computer connected to a network, with the sole purpose of supplying filebased data storage services to other devices on the network. The operating system and othersoftware on the NAS unit provide the functionality of data storage, file systems, and access tofiles, as well as the management of these functionalities.Over SubscriptionThe ratio of bandwidth required to bandwidth available. The switch is oversubscribed when allports, associated pair-wise, in any random fashion, cannot sustain full duplex at full line-rate.Port FencingPort fencing is a policy-based feature that allows you to protect your SAN from repeatedoperational or security problems experienced by switch ports. Port fencing allows you to setthreshold limits on the number of specific port events permitted during a given time period. Ifthe port generates more events during the specified time period, the Connectrix Manager(Port fencing feature) blocks the port, disabling transmit and allows you to receive traffic untilyou have time to investigate, solve the problem, and manually unblock the port.Principal SwitchPrincipal switch in a multiswitch fabric; the switch that allocates domain IDs to itself and to allother switches in the fabric. There is always one principal switch in a fabric. If a switch is notconnected to any other switches, it acts as its own principal switch.SANA storage area network (SAN) is an architecture to attach remote computer storage devices(such as disk arrays, tape libraries, and optical jukeboxes) to servers in such a way that thedevices appear locally attached to the operating system. Although SAN cost and complexityis dropping, they are still uncommon outside larger enterprises.VSANAn allocation of switch ports that can span multiple physical switches, forming a virtual fabric.A single physical switch can sometimes host more than one VSAN.World Wide Node Name2010 EMC Proven Professional Knowledge Sharing12

A unique identifier, even on global networks. The WWN is a 64-bit numberXX:XX:XX:XX:XX:XX:XX:XX). The WWN contains an OUI which uniquely determines theequipment manufacturer. OUIs are administered by the Institute of Electronic and ElectricalEngineers (IEEE). The Fibre Channel environment uses two types of WWNs; a World WideNode Name (WWNN) and a World Wide Port Name (WWPN). Typically, the WWPN is usedfor zoning (path provisioning function).ZoneAn information object implemented by the distributed Nameserver(dNS) of a Fibre Channelswitch. A zone contains a set of members which are permitted to discover and communicatewith one another. The members can be identified by a WWPN or port ID. EMC recommendsthe use of WWPNs in zone management. Zoning allows an administrator to group severaldevices by function or by location. All devices connected to a connectivity product, such as aConnectrix switch, may be configured into one or more zones.Zone SetAn information object implemented by the distributed Nameserver(dNS) of a Fibre Channelswitch. A Zone Set contains a set of Zones. A Zone Set is activated against a fabric, and onlyone Zone Set can be active in a fabric.2010 EMC Proven Professional Knowledge Sharing13

A.Designing SAN1.SAN Topology ConsiderationsThe adoption of SANs is driven by a variety of objectives. Some examples are:The need for more efficient use of enterprise storage arraysDecreasing size of backup/restore windowsIncreasing size of data set to be backed upThe need for improved high availability and disaster tolerance solutionsThe need to enhance storage resource managementSAN design can appear to be a challenging task, due to the large number of variablesinvolved in picking an appropriate design strategy. Designing a fabric involves many variablesthat require consideration. With each variable consideration comes a separate designdecision that must be made. Each design decision will help you create a fabric design that isappropriate for your business information model. The following parameters will help inchoosing the right SAN topology according to the requirements.a. AccessibilityAccessibility refers to the ability of your hosts to access the storage that is required to servicetheir applications. Accessibility can be measured by your ability to physically connect andcommunicate with the individual storage arrays, as well as your ability to provide enoughbandwidth resources to meet your full-access performance requirements. A storage array thatis physically accessible, but cannot be accessed within accepted performance limits becauseof oversaturated paths to the device, may be just as useless as an array that cannot bereached physically.An example of a statistical bandwidth network is the telephone system. The telephone systemis not constructed with enough bandwidth resources to allow every subscriber tocommunicate simultaneously. You may have heard "all lines are currently busy; please tryyour call again later." This message indicates that the number of subscribers has saturatedthe bandwidth currently available, so no new connections are possible until resources arefreed.Similar issues can arise in the design and implementation of a fabric. You should alsoconsider the internal design of the switching devices used in your fabric when considering2010 EMC Proven Professional Knowledge Sharing14

accessibility. While switches may be designed for high levels of connectivity and allow manyphysical attachments, their internal designs may cause internal bandwidth congestion.b. AvailabilityAvailability is a measurement of the amount of time that your data can be accessed,compared to the amount of time the data is not accessible because of issues in theenvironment. Lack of availability might be a result of failures in the environment that cause atotal loss of paths to the device, or it might be an event that caused so much bandwidthcongestion that the access performance renders the device virtually unavailable. Availabilityis impacted not only by your choice of components used to build the fabric, but also by yourability to build redundancy into the environment.Another concept that adds to the availability of an environment is sparing, which is theprocess of dedicating resources to remain unused until they are needed to take the place of afailed resource. The following must be considered in your redundancy and sparing plan: How much bandwidth do I need to preserve after a single event occurs? What other applications might be affected when the original storage resources move to a newpath or down to a single path? Do I need to plan for scenarios that include successive failures? Do I want redundancy built into my connectivity components (as seen with director-classswitching devices)? Do I want to build site redundancy and copy data to another site using CLARiiONMirrorView ? Do I want to build redundancy at the host level with a load-balancing and paths failoverapplication (like PowerPath )? How do I rank my business applications so that I can identify lower priority tasks, so theseresources can be used as spares during a failure event? An example of this would be if task onehad failed due to all of its fiber links being damaged and fiber links from task two were used tobring up the resources associated with task one. When the resources were back online, bothtasks would be working at 50 percent efficiency.2010 EMC Proven Professional Knowledge Sharing15

c. Resource ConsolidationResource consolidation includes the concepts of both physical and logical consolidation.Physical consolidation involves the physical movement of resources to a centralized location.Now that these resources are located together, you may be able to more efficiently usefacility resources, such as HVAC (heating, ventilation, and air conditioning), power protection,personnel, and physical security. The trade-off that comes with physical consolidation is theloss of resilience against a site failure. Flexibility is a measure of how rapidly you are able todeploy, shift, and redeploy new storage and host assets in a dynamic fashion withoutinterrupting your currently running environment. An example of flexibility is the ability tosimply connect new storage into the fabric and then zone it to any host in the fabric.d. SecuritySecurity refers to the ability to protect your operations from external and internal maliciousintrusions, as well as the ability to protect accidental or unintentional data access byunauthorized parties. Security can range from restriction of physical access to the servers,storage, and switches by placing them in a locked room, to logical security associated withzoning, volume accessing/ masking.e. SupportabilitySupportability is the measure of how easy it is to effectively identify and troubleshoot issues,as well as to identify and implement a viable repair solution in the environment. The ability totroubleshoot may be enhanced through good fabric designs, purposeful placement of serversand storage on the fabric, and a switch's ability to identify and report issues on the switchitself or in the fabric. Fabric topologies can be designed so that data traffic patterns aredeterministic, traffic bandwidth requirements can be easily associated with individualcomponents, and placement policies can be documented so that troublesome componentscan be identified quickly.2.SAN TopologiesThis section describes the SAN topologies that can be considered for designing a SANaccording to the importance of parameters explained above. This will help us logically decidewhich topology best suits and meets our requirements.2.a Simple Fibre Channel SAN topologiesA simple Fibre Channel SAN consists of less than four directors and switches connected byISLs and which has no more than two hops. A single switch fabric is the simplest of the2010 EMC Proven Professional Knowledge Sharing16

simple Fibre Channel SAN topologies and consists of only a single switch. A simple FibreChannel SAN consists of less than four directors or Fibre Channel Switches.Fibre SwitchStorageServerManagement StationFigure 4. Single Switch TopologyMost Å --------------------Æ yTable 1: Single Switch Subjective RatingThe following best practices are specific for two switch fabrics.ISL subscription best practice — While planning the SAN, keep track of how manyhost and storage pairs utilize the ISLs between domains. As a general best practice,if two switches are connected by ISLs, ensure that there is a minimum of two ISLsbetween them and that there are no more than six initiator and target pairs per ISL.For example, if 14 initiators access a total of 14 targets between two domains, a totalof three ISLs would be necessary. This best practice should not be applied blindlywhen setting up a configuration. Consider the applications that will use the ISLs.2.b Complex Fibre Channel SAN topologies2010 EMC Proven Professional Knowledge Sharing17

2.b.a Full-Mesh fabricA full-mesh fabric is any collection of Fibre Channel switches in which each switchis connected to every other switch in the fabric by one or more ISLs. For best hostand storage accessibility, it is recommended that a full-mesh fabric contain nomore than four switches. A mesh may contain departmental switches, Whendesigningandimplementing a full-mesh fabric, it is recommended that you lay out the storageand servers in a single-tier logical topology design and plan your ISLrequirements based on the assumption that 50% of the traffic on any one switchwill remain local and the other 50% will originate from the remaining remoteswitches.Figure 5. Full-Mesh TopologyBenefitsFull-mesh configurations give you, at most, one-hop access from any server to any storagedevice on the fabric. This means that when you are adding or migrating storage or serverattachments, you have the greatest possibility of placing the server attachment and matchingstorage attachments anywhere in the fabric and achieving the same response time. Meshesalso ensure that you always have multiple local and remote paths to the data even after fabricevents have occurred.LimitationsScaling a full-mesh solution becomes complicated and costly when increasing the number ofswitches and required ISLs to guarantee traffic performance.Most Å ----------------Æ LeastAttribute54321Accessibility2010 EMC Proven Professional Knowledge Sharing18

uritySupportabilityTable 2: Full-Mesh Switch Subjective Rating2.b.b Partial Mesh fabricFigure 6. Partial-Mesh TopologyA partial-mesh fabric is different from a full mesh in that each switch does not have to beconnected to all other switches. However, to be considered a partial mesh, the fabric must bea configuration where splitting it results in each new sub-fabric being a full mesh. For bestfabric response times, both the managed switch (where zoning is activated) and the principalswitch should be at the logical center of the fabric.BenefitsPartial-mesh designs offer extensive access to both local switch storage and single-hopstorage. A partial mesh also extends accessibility and provides many unique paths to thestorage. Increasing accessibility while maintaining the same level of robustness is a designgoal for every topology. Partial meshes also offer a simple progression into a core/edgedesign. If you look at the center of the partial mesh as the core, you can create the newinfrastructure by simply removing some of the ISLs at the outer edges of the fabric.Limitations2010 EMC Proven Professional Knowledge Sharing19

Increasing the fabric size always increases the dependencies within the fabric. This does notcause a problem, but it does increase the complexity of troubleshooting and the impact onunrelated processes during a fabric event.Most Å ----------------Æ yTable 3: Partial-Mesh Switch Subjective Rating2.b Core Edge Fibre Channel SAN topologies1. Within the two-tier design, servers connect to the edge switches, and storage devicesconnect to one or more core switches. This allows the core switch to provide storage servicesto one or more edge switches, thus servicing more servers in the fabric. The interswitch links(ISLs) will have to be designed so that the overall fabric maintains both the fan-out ratio ofservers to storage and the overall end-to-end oversubscription ratio.2. Three-tier: Edge-core-edge designA three-tier design may be ideal in environments where future network growth will result inthe number of storage devices exceeding the number of ports available at the core switch.This type of topology still uses a set of edge switches for server connectivity, but addsanother set of edge switches for storage devices. Both sets of edge switches connect to acore switch via ISLs.2010 EMC Proven Professional Knowledge Sharing20

Figure 7. Core Edge FabricBenefitsThe compound core/edge model maintains a robust, highly efficient traffic model whilereducing the required ISLs, thus increasing the available ports for both storage and hostattachments. It also offers a simple method for the expansion of two or more simplecore/edge fabrics into a single environment. You can easily create a compound core topologyby connecting the core switches from simple core/edge fabrics into a full mesh. Thecompound core/edge topology creates a robust back-end fabric that can extend theopportunities for sharing of both backup and storage resources.LimitationsCore/edge design models produce a physically larger, tiered fabric which could result inslightly longer fabric management propagation times over smaller, more compact designs.Neither compound nor complex core/edge fabrics provide for single-hop access to all storage.Most Å --------------------Æ idationFlexibilityScalability2010 EMC Proven Professional Knowledge Sharing21

SecuritySupportabilityTable 4: Core Edge Subjective RatingBest Practices for Core Edge TopologyLay out the host and storage connectivity such that if a switch fails, not all of aparticular hosts’ storage becomes inaccessible.The use of two separate management networks is more common with balancedfabrics, but it can still be employed when only one fabric is used.ISL subscription best practice — While planning the SAN, keep track of the number ofhost and storage pairs that would be utilizing the ISLs between domains. As ageneral best practice, if two switches are connected by ISLs, ensure that ther

Switch and Zoning Best Practices 28-30 2. IP SAN Best Practices 30-32 3. RAID Group Best Practices 32-34 4. HBA Tuning 34-38 5. Hot Sparing Best Practices 38-39 6. Optimizing Cache 39 7. Vault Drive Best Practices 40 8. Virtual Provisioning Best Practices 40-43 9. Drive