Cisco MDS 9000 Family Cookbook For MDS SAN-OS Release 2
Cisco MDS 9000 Family Cookbookfor SAN-OS 2.xSeth MasonVenkat Kirishnamurthyi4/25/06Corporate HeadquartersCisco Systems, Inc.170 West Tasman DriveSan Jose, CA 95134-1706USAhttp://www.cisco.comTel: 408 526-4000800 553-NETS (6387)Fax: 408 526-4100
CCSP, the Cisco Square Bridge logo, Cisco Unity, Follow Me Browsing, FormShare, and StackWise are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live,Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, Cisco, the CiscoCertified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Empowering the Internet Generation,Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, GigaDrive, GigaStack, HomeLink, Internet Quotient, IOS, IP/TV, iQ Expertise, the iQ logo, iQ Net ReadinessScorecard, LightStream, Linksys, MeetingPlace, MGX, the Networkers logo, Networking Academy, Network Registrar, Packet, PIX, Post-Routing, Pre-Routing, ProConnect,RateMUX, Registrar, ScriptShare, SlideCast, SMARTnet, StrataView Plus, SwitchProbe, TeleRouter, The Fastest Way to Increase Your Internet Quotient, TransPath, and VCO areregistered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationshipbetween Cisco and any other company. (0406R)Cisco MDS 9000 Family CookbookCopyright 2004 Cisco Systems, Inc. All rights reserved.Comments: mds-cookbook@cisco.com
CONTENTSPrefaceixAudienceixAbout the AuthorsOrganizationixxDocument ConventionsxRelated DocumentationxiObtaining Documentation xiiCisco.com xiiProduct Documentation DVD xiiiOrdering Documentation xiiiDocumentation FeedbackxiiiCisco Product Security Overview xivReporting Security Problems in Cisco ProductsxivObtaining Technical Assistance xvCisco Technical Support & Documentation WebsiteSubmitting a Service Request xvDefinitions of Service Request Severity xviObtaining Additional Publications and InformationCHAPTER1Managing a Cisco MDS 9000 Switchxvxvi1-1Using SNMP to Monitor the MDS 1-1Events 1-1Thresholds 1-2Third Party Management Application ConfigurationAdvanced Cisco MDS Monitoring 1-81-3CFS: Cisco Fabric Services 1-8Fabric Manager and CFS 1-10How does this work? 1-11CFS CLI Commands 1-11Which switches are CFS capable? 1-11What CFS applications do I have and what is their scope?Why am I locked out of an application by CFS? 1-12Command Scheduler 1-13Automated Switch Configuration Backup1-121-13Cisco MDS 9000 Family Cookbook for SAN-OS 2.xOL-xxxxx-xxiii
ContentsCopying Files to and from a Switch 1-16Copying Files Using the CLI 1-16Secure Copy Protocol 1-16Secure File Transfer Protocol 1-17Managing Files on the Standby Supervisor 1-18Delete a File from the Standby Supervisor 1-18Firmware Upgrades and Downgrades 1-20Upgrading firmware with the CLI 1-20Downgrading Firmware with the CLI 1-22Upgrading Firmware with Fabric Manager 1-23Password Recovery1-26Installing a License 1-29Using the CLI to Install a License: 1-29Using Fabric Manager to Install a License 1-30Which Feature is Enabling the License Grace Period?Check with Fabric Manager 1-33Check with the CLI 1-33Copying Core Files From Switch1-331-34Restoring a Fixed Switch Configuration1-35Configuring an NTP Server 1-38Configuring NTP with CFS 1-38Configure NTP without CFS 1-39What to do Before Calling TAC1-41Saving the Configuration Across the FabricHow to Disable the Web Server1-431-44Device Aliases 1-45Manipulating Device Aliases with the CLI 1-46Displaying Device Aliases with the CLI 1-46Creating Device Aliases with the CLI 1-46Converting FC Aliases to Device Aliases 1-47Device Aliases with Fabric Manager 1-48Enabling Fabric Manager to use Device Aliases 1-49Creating a Device Alias for an Existing Device 1-50Creating a Device Alias for a New Device 1-51Implementing Syslog1-52Configuring Call Home 1-54What are Alert Groups? 1-54Configure Call Home to Send All Notifications to a Single E-Mail Address1-55Cisco MDS 9000 Family Cookbook for SAN-OS 2.xivOL-xxxxx-xx
ContentsManaging Fabric Manager 1-58Operating Fabric Manager Through a Firewall using SNMP Proxy 1-58Configuration using a non-NAT Packet Filter 1-58Performance Manager (PM) using Fabric Manager Server (FMS) 1-60Launching Performance Manager Configuration from a Host Running FMSCHAPTER2Account Management1-602-1Creating User Accounts2-2Creating a User Role 2-3Creating a Role with Device ManagerCreating a Role with CLI 2-72-4Configuring TACACS with Cisco SecureACS 2-9Authentication and Authorization with TACACS 2-9Configure SecureACS Server 2-10Configure TACACS on the MDS Switch 2-14Accounting with TACACS 2-15Configuring the MDS Switch 2-16Configuring SecureACS 2-16Providing Password-free Access Using SSHCHAPTER3Physical Interfaces2-193-1Configuring FC ports 3-1Port Description 3-1Port Speed 3-1Port Mode Auto 3-2Port Mode E 3-2Port Mode F 3-2Port Mode FL 3-2Port Mode Fx 3-3Port Mode SD 3-3Port mode ST 3-3Port mode TL 3-3Configuring Trunking E ports 3-4Trunk Port Mode 3-4Configuring Trunk Ports to Filter Specific VSANsEnabling Port Beaconing 3-4Configuring Gigabit Ethernet PortsConfiguring VRRP 3-53-43-5Implementing WWN Based VSANs (DPVM)3-7Cisco MDS 9000 Family Cookbook for SAN-OS 2.xOL-xxxxx-xxv
ContentsAdding Existing Devices to DPVM 3-9Adding New Devices to DPVM 3-11Modify the VSAN Assignment of a DPVM EntryDPVM Conflicting Entries 3-14DPVM with the CLI 3-16Adding Existing Devices to DPVM 3-16Adding New Devices to DPVM 3-17Modify the VSAN Assignment of a DPVM EntryCHAPTER4Logical Interfaces3-133-184-1Port Channels 4-1Quiesce a Port Channel or ISL Link 4-1Creating a Port Channel using FM 4-2Creating a Port Channel using CLI 4-5Adding a New Member to a Port Channel (FM) 4-7Adding New Members to a Port Channel (CLI) 4-9Modifying the VSAN Allowed List on a Port Channel (FM) 4-10Modifying the VSAN Allowed List on a port channel (CLI) 4-10CHAPTER5VSANs5-1Creating a VSAN and Adding Interfaces5-1Modifying VSAN Attributes with Fabric Manager 5-3Converting an Existing VSAN DomainID and Enabling FCID with Fabric Manager5-4Modifying VSAN Attributes with the CLI 5-7Creating a VSAN on a single switch and adding an Interface 5-7Setting VSAN Interop Mode 5-7Interop Mode 1 5-8Interop Mode 2 5-8Interop Mode 3 5-8Changing the Load-balancing Scheme 5-8Sequence Level load-balancing (Source ID, Destination ID) 5-8Exchange level load balancing (S ID, D ID, OX ID) 5-9Converting an Existing VSAN to Static DomainID and Enabling Persistent FCID using CLIRestarting a VSAN 5-10Assigning a Predetermined FCID to a PWWN 5-10CHAPTER6Zoning5-96-1Enhanced Zoning 6-1Enabling Enhanced Zoning6-2Cisco MDS 9000 Family Cookbook for SAN-OS 2.xviOL-xxxxx-xx
ContentsEnabling Enhanced Zoning with the CLI 6-3Enabling Enhanced Zoning with Fabric Manager 6-3Displaying User with Current Lock in CLI and Fabric Manager6-4Zone Sets 6-6Distributing Zone Sets 6-6Distributing Zone Sets Automatically 6-6Distributing Zone Sets Manually 6-7Zones 6-8Creating a Zone and Adding it to a Zone Set with Fabric Manager 6-8Creating Non-pWWN Based Zones 6-13Creating a Zone and Adding it to a Zone Set with the CLI Standalone Method 6-14Creating a Zone and Adding it to a Zone Set with the CLI Inline Method 6-15Creating a FC Alias-based Zone with the CLI 6-16Creating an Interface-based Zone with the CLI 6-18CHAPTER7Inter-VSAN Routing7-1IVR Core Components 7-2IVR Topology 7-2Auto-Topology 7-2Transit VSANs 7-3Configuring a Three Switch, Two Transit VSAN Topology with CFSIVR Zones and Zone Sets 7-7IVR with CFS 7-87-4IVR-1 7-10Enabling IVR-1 7-10Enabling IVR-1 with the CLI 7-10Enabling IVR-1 with Fabric Manager 7-11Configuring a Single Switch and Two VSANs 7-12Creating the IVR Topology 7-12Creating the IVR Zone Set and Zones 7-13IVR-2 with FC-NAT 7-16Enabling IVR-2 7-16Upgrading from IVR-1 to IVR-2 7-19Configuring Persistent FC IDs in IVR 7-21Configuring a Single Switch with Two VSANsAdding a New IVR Enabled Switch 7-27CHAPTER8FCIP7-238-1Enabling FCIP8-1Cisco MDS 9000 Family Cookbook for SAN-OS 2.xOL-xxxxx-xxvii
ContentsConfiguring FCIP on a Switch with CLI8-2Configuring Multiple FCIP Tunnels Using a Single gigE portConfiguring FCIP with IPsec using FM8-78-15Tuning FCIP 8-24TCP Tuning: Latency and Available Bandwidth 8-24Enabling FCIP Write Acceleration 8-25Enabling FCIP Compression 8-26Enabling Tape Acceleration 8-27Enabling Tape Acceleration from the CLI 8-27Enabling Tape Acceleration from the CLI 8-30Testing and Tuning the FCIP link with SETCHAPTER9iSCSI8-319-1Enabling iSCSI9-1Configuring iSCSI on an MDS Switch in Transparent ModeConfiguring iSCSI on the MDS Switch in Proxy initiator mode9-29-7Configuring iSCSI Client Initiators on Hosts 9-12Configuring iSCSI on Microsoft Windows 9-12Configuring an iSCSI Client on Linux 9-16Cisco MDS 9000 Family Cookbook for SAN-OS 2.xviiiOL-xxxxx-xx
PrefaceThis document adresses the configuration and implementation of fabrics using Cisco’s MDS 9000Family of Fibre Channel Switch and Director Class products. The configuration procedures andcomponents provided have been tested and validated by Cisco’s Solution-Interoperability Engineeringdepartment.This cookbook provides simplified, concise recipes (procedures) for tasks that might be required toconfigure a Cisco MDS 9000 switch. This guide does not replace the MDS 9000 Family ConfigurationGuides.AudienceThis document is designed for use by Cisco TAC, Sales, Support Engineers, Professional ServicePartners, Systems Administrators and others responsible for the design and deployment of Storage AreaNetworks in the data center environment.This is field-driven book, meaning that the intended audience (the storage administrators, technicalsupport engineers, SEs and CEs) is also the source of information for these procedures. Theirrequirements for a procedure are what determine the content.If there are procedures that you feel should be covered in this book, or if you have any other commentsor questions, please notify us via email at: mds-cookbook@cisco.com. Please state the document name,page number, and details of the request.About the AuthorsSeth Mason is a Network Consulting Engineer with the DCN team at Cisco Systems. His areas ofexpertise are SAN migration, Disaster Recovery, interoperability and InterVSAN Routing. He graduatedfrom Auburn University in 1998 with a Bachelor of Computer Engineering and has focused on SANsever since, including Product Engineer with IBM’s Storage Subsystems Group, Silicon ValleyOperations team lead with StorageNetworks and NCE with Andiamo Systems. Seth has continued tofurther his expertise in storage by authoring both the MDS-9000 Family Cookbook for SAN-OS 1.x andMDS-9000 Family Cookbook for SAN-OS 2.x as well as the MDS-9000 Switch to SwitchInteroperability Configuration Guide, and is a member of the team that authored the CCIE exam inStorage Networking.Cisco MDS 9000 Family Cookbook for SAN-OS 2.xOL-xxxxx-xxix
PrefaceOrganizationVenkat Kirishnamurthyi is a Network Consulting Engineer with the DCN team at Cisco Systems. Hisareas of expertise are SAN design, migration, and storage replication for disaster recovery. He graduatedfrom Bangalore University in 1992 with a Bachelor of Electronics and Communications Engineering.Since then he has worked as a Systems Administrator at Hughes Software Systems India and as a Sr.Systems Administrator and Sr. Storage Administrator at Cisco Systems. Venkat has continued hisstorage expertise by authoring SAN migration guides for HPUX and Solaris hosts, both the MDS-9000Family Cookbook for SAN-OS 1.x and MDS-9000 Family Cookbook for SAN-OS 2.x. He is a memberof the team that authored the CCIE exam for Storage Networking.OrganizationThis guide is organized as follows:ChapterTitleContent DescriptionChapter 1Managing a Cisco MDS 9000 Switch Recipes for various aspects of managing theCisco MDS 9000 switches.Chapter 2Account ManagementProcedures for managing users and theiraccounts.Chapter 3Physical InterfacesProcedures for configuring the variousFibreChannel (FC) and Gigabit Ethernet portson Cisco MDS 9000 switches.Chapter 4Logical InterfacesProcedures for building, modifying andreducing a PortChannel.Chapter 5VSANsProcedures for creating and configuringVSANs.Chapter 6ZoningProcedures for creating and configuring zonesand zone sets.Chapter 7Inter-VSAN RoutingProcedures for configuring inter-VSANrouting.Chapter 8FCIPProcedures for creating and managing FCIPlinks between Cisco MDS 9000 switches.Chapter 9iSCSIProcedures for configuring iSCSI on CiscoMDS 9000 switches.Document ConventionsCommand descriptions use these conventions:boldface fontCommands and keywords are in boldface.italic fontArguments for which you supply values are in italics.[ ]Elements in square brackets are optional.[x y z]Optional alternative keywords are grouped in brackets and separated byvertical bars.Cisco MDS 9000 Family Cookbook for SAN-OS 2.xxOL-xxxxx-xx
PrefaceRelated DocumentationScreen examples use these conventions:screen fontTerminal sessions and information the switch displays are in screen font.boldface screen fontInformation you must enter is in boldface screen font.italic screen fontArguments for which you supply values are in italic screen font. Nonprinting characters, such as passwords, are in angle brackets.[ ]Default responses to system prompts are in square brackets.!, #An exclamation point (!) or a pound sign (#) at the beginning of a line of codeindicates a comment line.This document uses the following conventions:NoteCautionTipMeans reader take note. Notes contain helpful suggestions or references to material not covered in themanual.Means reader be careful. In this situation, you might do something that could result in equipmentdamage or loss of data.Refers to a best practice for implementing the Cisco MDS 9000 platform. Tips are based on in-depthknowledge of the platform, as well as extensive experience implementing SANs.Related DocumentationThe documentation set for the Cisco MDS 9000 Family includes the following documents: Cisco MDS 9000 Family Release Notes for Cisco MDS SAN-OS Releases Cisco MDS 9000 Family Interoperability Support Matrix Cisco MDS SAN-OS Release Compatibility Matrix for IBM SAN Volume Controller Software forCisco MDS 9000 Cisco MDS SAN-OS Release Compatibility Matrix for VERITAS Storage Foundation for NetworksSoftware Cisco MDS SAN-OS Release Compatibility Matrix for Storage Service Interface Images Cisco MDS 9000 Family SSM Configuration Note Cisco MDS 9000 Family ASM Configuration Note Regulatory Compliance and Safety Information for the Cisco MDS 9000 Family Cisco MDS 9500 Series Hardware Installation Guide Cisco MDS 9200 Series Hardware Installation GuideCisco MDS 9000 Family Cookbook for SAN-OS 2.xOL-xxxxx-xxxi
PrefaceObtaining Documentation Cisco MDS 9216 Switch Hardware Installation Guide Cisco MDS 9100 Series Hardware Installation Guide Cisco MDS 9020 Fabric Switch Hardware Installation Guide Cisco MDS 9000 Family Software Upgrade and Downgrade Guide Cisco MDS 9000 Family Configuration Guide Cisco MDS 9000 Family Command Reference Cisco MDS 9020 Fabric Switch Configuration Guide and Command Reference Cisco MDS 9000 Family Fabric Manager Configuration Guide Cisco MDS 9000 Family Fabric and Device Manager Online Help Cisco MDS 9000 Family SAN Volume Controller Configuration Guide Cisco MDS 9000 Family Quick Configuration Guide Cisco MDS 9000 Family Fabric Manager Quick Configuration Guide Cisco MDS 9000 Family MIB Quick Reference Cisco MDS 9020 Fabric Switch MIB Quick Reference Cisco MDS 9000 Family CIM Programming Reference Cisco MDS 9000 Family System Messages Reference Cisco MDS 9020 Fabric Switch System Messages Reference Cisco MDS 9000 Family Troubleshooting Guide Cisco MDS 9000 Family Port Analyzer Adapter 2 Installation and Configuration Note Cisco MDS 9000 Family Port Analyzer Adapter Installation and Configuration NoteFor information on VERITAS Storage Foundation for Networks for the Cisco MDS 9000 Family, referto the VERITAS website: http://support.veritas.com/For information on IBM TotalStorage SAN Volume Controller Storage Software for the Cisco MDS9000 Family, refer to the IBM TotalStorage Support 00/Obtaining DocumentationCisco documentation and additional literature are available on Cisco.com. You can also obtain technicalassistance and other technical resources from Cisco Systems, as described below.Cisco.comAccess the most current Cisco documentation at:http://www.cisco.com/techsupportAccess the Cisco website at:http://www.cisco.comAccess international Cisco websites at:http://www.cisco.com/public/countries languages.shtmlCisco MDS 9000 Family Cookbook for SAN-OS 2.xxiiOL-xxxxx-xx
PrefaceDocumentation FeedbackProduct Documentation DVDCisco documentation and other literature is available in the Product Documentation DVD package,which may have shipped with your product. The Product Documentation DVD is updated regularly andmay be more current than printed documentation.The Product Documentation DVD is a comprehensive library of technical product documentation onportable media. The DVD enables you to access multiple versions of hardware and software installation,configuration, and command guides for Cisco products and to view technical documentation in HTML.With the DVD, you have access to the same documentation that is found on the Cisco website withoutbeing connected to the Internet. Certain products also have .pdf versions of the documents.The Product Documentation DVD is available as a single unit or as a subscription. Registered Cisco.comusers (Cisco direct customers) can order a Product Documentation DVD (product numberDOC-DOCDVD ) from the Ordering tool or Cisco Marketplace.Cisco Ordering isco dering DocumentationBeginning June 30, 2005, registered Cisco.com users may order Cisco documentation at the ProductDocumentation Store in the Cisco Marketplace at this URL:http://www.cisco.com/go/marketplace/Cisco will continue to support documentation orders using the ordering tool: Registered Cisco.com users (Cisco direct customers) can order documentation from theOrdering tool:http://www.cisco.com/en/US/partner/ordering/ Instructions for ordering documentation using the Ordering tool are at this URL:http://www.cisco.com/univercd/cc/td/doc/es inpck/pdi.htm Nonregistered Cisco.com users can order documentation through a local account representative bycalling Cisco Systems Corporate Headquarters (California, USA) at 408 526-7208 or, elsewhere inNorth America, by calling 1 800 553-NETS (6387).Documentation FeedbackYou can rate and provide feedback about Cisco technical documents by completing the online feedbackform that appears with the technical documents on Cisco.com.You can send comments about Cisco documentation to bug-doc@cisco.com.Cisco MDS 9000 Family Cookbook for SAN-OS 2.xOL-xxxxx-xxxiii
PrefaceCisco Product Security OverviewYou can submit comments by using the response card (if present) behind the front cover of yourdocument or by writing to the following address:Cisco SystemsAttn: Customer Document Ordering170 West Tasman DriveSan Jose, CA 95134-9883We appreciate your comments.Cisco Product Security OverviewCisco provides a free online Security Vulnerability Policy portal at this URL:http://www.cisco.com/en/US/products/products security vulnerability policy.htmlFrom this site, you can: Report security vulnerabilities in Cisco products. Obtain assistance with security incidents that involve Cisco products. Register to receive security information from Cisco.A current list of security advisories and notices for Cisco
This cookbook provides simplified, concise recipes (procedures) for tasks that might be required to configure a Cisco MDS 9000 switch. This guide does not replace the MDS 9000 Family Configuration Guides. Audience This document is designed for use by Cisco TAC, Sales, Support Engineers, Professional Service
Cisco MDS 9000 Family Hardware and NX-OS Release 5.x Supported Software 1-2 Cisco MDS 9000 Family Hardware and NX-OS Release 4.2x Supported Software 1-8 Cisco MDS 9000 Family Hardware and NX-OS Release 4.1x Supported Software 1-15 Cisco MDS 9000 Family Hardware
Which Cisco MDS model supports the most Fibre Channel ports per chassis? A. MDS 9513 B. MDS 9509 C. MDS 9506 D. MDS 9250i Correct Answer: A QUESTION 20 Refer to the exhibit. Which Cisco MDS chassis supports the 48-Port 16-Gbps Fibre Channel Switching Module? A. MDS 9509 B. MDS 9513 C. MDS 9710 D. MDS 9506 Correct Answer: C QUESTION 21 Refer to .
operating system running on the Cisco MDS 9000 family of Multilayer Directors and Fabric Switches. The SAN-OS software is the same base system software used throughout the entire Cisco MDS 9000 product line. The Cisco MDS 9000 family of switches provides the infrastructure that ties together file servers and back end storage.
Cisco MDS 9000 Family Command Reference 78-16088-01, Cisco MDS SAN-OS Release 1.3 New and Changed Information Table 1 summarizes the new and changed features for the Cisco MDS 9000 Family Command Reference, and tells you where they are documented. If a feature has changed in Release 1.3, a brief description of
Cisco Nexus 1000V Cisco Nexus 1010 Cisco Nexus 4000 Cisco MDS 9100 Series Cisco Nexus 5000 Cisco Nexus 2000 Cisco Nexus 6000 Cisco MDS 9250i Multiservice Switch Cisco MDS 9700 Series Cisco Nexus 7000/7700 Cisco Nexus 3500 and 3000 CISCO NX-OS: From Hypervisor to Core CISCO DCNM: Single
Cisco MDS 9509 CLI MDS 9000 CLI Cisco IOS CLI Cisco MDS 9000 CLI Cisco Fabric Manager Java Cisco Fabric Manager Cisco Fabric Manager API Supervisor Supervisor 1 1 SFP VSAN / 1/2Gbps 255 16 224 1/2Gbps 48 1
MDS 9000 Fabric Switch Positioning Cisco positioned to extend reach all market segments IP Storage Services – iSCSI and FCIP MDS 9216 and 9216i 14 -Port, 16 Port, 32-Port 1 & 2 Gb FC MDS 9020* * FabricWare OS Cisco Fabric Manager Cisco MDS 9000 Family SAN -OS MDS 9509 4-Port 10Gb FC 12 Port, 24 48-Port 1, 2 & 4Gb FC Small/Medium Business .
America’s criminal justice system. Racial and ethnic disparity foster public mistrust of the criminal jus-tice system and this impedes our ability to promote public safety. Many people working within the criminal justice system are acutely aware of the problem of racial disparity and would like to counteract it. The pur-pose of this manual is to present information on the causes of disparity .
