Brocade 7.1.0 Fabric OS Troubleshooting And Diagnostics Guide

Brocade configuration form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Chapter 5Firmware Download ErrorsBlade troubleshooting tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Firmware download issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Troubleshooting with the firmwareDownload command . . . . . . . . . 54Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . 55USB error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55Considerations for downgrading firmware . . . . . . . . . . . . . . . . . . . . 56Preinstallation messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56Blade types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Firmware versions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Chapter 6SecurityPasswords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61Password recovery options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62Device authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62Protocol and certificate management . . . . . . . . . . . . . . . . . . . . . . . . 63Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . 63SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . 64FIPS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Chapter 7Virtual FabricsGeneral Virtual Fabrics troubleshooting . . . . . . . . . . . . . . . . . . . . . . 65Fabric identification issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Logical Fabric issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Base switch issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67Logical switch issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Switch configuration blade compatibility. . . . . . . . . . . . . . . . . . . . . . 69Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70Chapter 8ISL TrunkingLink issues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71Buffer credit issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72Getting out of buffer-limited mode . . . . . . . . . . . . . . . . . . . . . . . 72Fabric OS Troubleshooting and Diagnostics Guide53-1002751-01vii

Chapter 9ZoningOverview of corrective action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73Verifying a fabric merge problem . . . . . . . . . . . . . . . . . . . . . . . . 73Verifying a TI zone problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73Segmented fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Zone conflicts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Correcting a fabric merge problem quickly . . . . . . . . . . . . . . . . 76Changing the default zone access . . . . . . . . . . . . . . . . . . . . . . . 77Editing zone configuration members . . . . . . . . . . . . . . . . . . . . . 77Reordering the zone member list . . . . . . . . . . . . . . . . . . . . . . . . 77Checking for Fibre Channel connectivity problems . . . . . . . . . . 78Checking for zoning problems. . . . . . . . . . . . . . . . . . . . . . . . . . . 79Gathering additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80Chapter 10Diagnostic FeaturesAbout Fabric OS diagnostics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81Diagnostic information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81Power-on self test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82Disabling POST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84Enabling POST . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84Switch status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84Viewing the overall status of the switch . . . . . . . . . . . . . . . . . . . 84Displaying switch information . . . . . . . . . . . . . . . . . . . . . . . . . . . 85Displaying the uptime for a switch . . . . . . . . . . . . . . . . . . . . . . . 86Using the SpinFab and portTest commands . . . . . . . . . . . . . . . . . . . 87Debugging spinFab errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88Clearing the error counters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89Enabling a port. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89Disabling a port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Diagnostic Port (D Port) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Understanding D Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91Supported topologies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93Using D Port without HBAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95Using D Port with HBAs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97Controlling testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100Example test scenarios and output . . . . . . . . . . . . . . . . . . . . .100Port information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .105Viewing the status of a port . . . . . . . . . . . . . . . . . . . . . . . . . . .105Displaying the port statistics. . . . . . . . . . . . . . . . . . . . . . . . . . .106Displaying a summary of port errors for a switch . . . . . . . . . . 107Equipment status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .108Checking the temperature, fan, and power supply . . . . . . . . .108Checking the status of the fans . . . . . . . . . . . . . . . . . . . . . . . .108Checking the status of a power supply. . . . . . . . . . . . . . . . . . .109Checking temperature status . . . . . . . . . . . . . . . . . . . . . . . . . .109viiiFabric OS Troubleshooting and Diagnostics Guide53-1002751-01

System message log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .110Displaying the system message log, with no page breaks . . .110Displaying the system message logone message at a time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111Clearing the system message log . . . . . . . . . . . . . . . . . . . . . . .111Port log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111Viewing the port log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111Syslogd configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113Configuring the host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113Configuring the switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .114Automatic trace dump transfers . . . . . . . . . . . . . . . . . . . . . . . . . . .115Specifying a remote server . . . . . . . . . . . . . . . . . . . . . . . . . . . .115Enabling the automatic transfer of trace dumps. . . . . . . . . . .115Setting up periodic checking of the remote server . . . . . . . . .115Saving comprehensive diagnostic files to the server . . . . . . .116Appendix ASwitch Type and Blade IDAppendix BHexadecimal ConversionExample conversion of the hexadecimal triplet Ox616000 . .121Decimal-to-hexadecimal conversion table . . . . . . . . . . . . . . . .122IndexFabric OS Troubleshooting and Diagnostics Guide53-1002751-01ix

xFabric OS Troubleshooting and Diagnostics Guide53-1002751-01

About This DocumentIn this chapter How this document is organized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Supported hardware and software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii What’s new in this document . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii Document conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv Getting technical help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Document feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviHow this document is organizedThe document contains the following chapters: Chapter 1, “Introduction,” gives a brief overview of troubleshooting the Fabric OS, and providesprocedures for gathering basic information from your switch and fabric to aid introubleshooting. Chapter 2, “General,” provides information on licensing, hardware, and syslog issues. Chapter 3, “Connectivity,” provides information and procedures on troubleshooting various linkissues. Chapter 4, “Configuration,” provides troubleshooting information and procedures forconfiguration file issues. Chapter 5, “Firmware Download Errors,” provides procedures for troubleshooting firmwaredownload issues. Chapter 6, “Security,” provides procedures for user account and security issues.Chapter 7, “Virtual Fabrics,” provides procedures for troubleshooting Virtual Fabrics.Chapter 8, “ISL Trunking,” provides procedures for resolving trunking issues.Chapter 9, “Zoning,” provides preparations and procedures for performing firmwaredownloads, as well troubleshooting information. Chapter 10, “Diagnostic Features,” provides procedures for the use of the diagnosticscommands for the chassis, ports, and other chassis equipment. Provides information on thesystem messages. Appendix A, “Switch Type and Blade ID,” provides reference information to guide you inunderstanding switch output. Appendix B, “Hexadecimal Conversion,” provide reference information for translatinghexadecimal output.Fabric OS Troubleshooting and Diagnostics Guide53-1002751-01xi

Supported hardware and softwareIn those instances in which procedures or parts of procedures documented here apply to someswitches but not to others, this guide identifies which switches are supported and which are not.Although many different software and hardware configurations are tested and supported byBrocade Communications Systems, Inc. for Fabric OS v7.1.0, documenting all possibleconfigurations and scenarios is beyond the scope of this document.The following hardware platforms are supported by this release of Fabric OS: Brocade 300 switchBrocade 5100 switchBrocade 5300 switchBrocade 5410 embedded switchBrocade 5424 embedded switchBrocade 5450 embedded switchBrocade 5460 embedded switchBrocade 5470 embedded switchBrocade 5480 embedded switchBrocade 6505 switchBrocade 6510 switchBrocade 6520 switchBrocade 7800 extension switchBrocade 8000 FCoE switchBrocade VA-40FCBrocade Encryption SwitchBrocade DCXBrocade DCX-4SBrocade DCX 8510-4Brocade DCX 8510-8What’s new in this documentUpdated for Brocade Fabric OS v7.1.0, including the following: Updated system messages related to firmware downloads. (Refer to Chapter 5, “FirmwareDownload Errors,” on page 51.) Introduced new features available with the D Port diagnostic tool. (Refer to Chapter 10,“Diagnostic Features,” on page 81.)For further information about documentation updates for this release, refer to the release notes.xiiFabric OS Troubleshooting and Diagnostics Guide53-1002751-01

Document conventionsThis section describes text formatting conventions and important notice formats used in thisdocument.TEXT FORMATTINGThe narrative-text formatting conventions that are used are as follows:bold textIdentifies command namesIdentifies the names of user-manipulated GUI elementsIdentifies keywords and operandsIdentifies text to enter at the GUI or CLIitalic textProvides emphasisIdentifies variablesIdentifies paths and Internet addressesIdentifies document titlescode textIdentifies CLI outputIdentifies command syntax examplesCOMMAND SYNTAX CONVENTIONSFor readability, command names in the narrative portions of this guide are presented in mixedlettercase: for example, switchShow. In actual examples, command lettercase is often alllowercase. Otherwise, this manual specifically notes those cases in which a command is casesensitive. Command syntax in this manual follows these conventions:commandCommands are printed in bold.--option, optionCommand options are printed in bold.-argument, argArguments.[]Optional element.variableVariables are printed in italics. In the help pages, values are underlined orenclosed in angled brackets .Repeat the previous element, for example “member[;member.]”valueFixed values following arguments are printed in plain font. For example,--show WWN Boolean. Elements are exclusive. Example: --show -mode egress ingressCOMMAND EXAMPLESThis book describes how to perform configuration tasks using the Fabric OS command lineinterface, but does not describe the commands in detail. For complete descriptions of all Fabric OScommands, including syntax, operand description, and sample output, refer to the Fabric OSCommand Reference.Fabric OS Troubleshooting and Diagnostics Guide53-1002751-01xiii

NOTES, CAUTIONS, AND WARNINGSThe following notices and statements are used in this manual. They are listed below in order ofincreasing severity of potential hazards.NOTEA note provides a tip, guidance, or advice, emphasizes important information, or provides areference to related information.ATTENTIONAn Attention statement indicates potential damage to hardware or data.CAUTIONA Caution statement alerts you to situations that can be potentially hazardous to you or causedamage to hardware, firmware, software, or data.DANGERA Danger statement indicates conditions or situations that can be potentially lethal or extremelyhazardous to you. Safety labels are also attached directly to products to warn of these conditionsor situations.KEY TERMSFor definitions specific to Brocade and Fibre Channel, refer to the Brocade Glossary.For definitions of SAN-specific terms, visit the Storage Networking Industry Association onlinedictionary nal informationThis section lists additional Brocade and industry-specific documentation that you might findhelpful.BROCADE RESOURCESTo get up-to-the-minute information, go to http://my.brocade.com and register at no cost for a userID and password.For practical discussions about SAN design, implementation, and maintenance, you can obtainBuilding SANs with Brocade Fabric Switches through:http://www.amazon.comxivFabric OS Troubleshooting and Diagnostics Guide53-1002751-01

White papers, online demonstrations, and data sheets are available through the Brocade s/products/index.pageFor additional Brocade documentation, visit the Brocade website:http://www.brocade.comRelease notes are available on the MyBrocade website and are also bundled with the Fabric OSfirmware.OTHER INDUSTRY RESOURCESFor additional resource information, visit the Technical Committee T11 website. This websiteprovides interface standards for high-performance and mass storage applications for FibreChannel, storage management, and other applications:http://www.t11.orgFor information about the Fibre Channel industry, visit the Fibre Channel Industry ing technical helpContact your switch support supplier for hardware, firmware, and software support, includingproduct repairs and part ordering. To expedite your call, have the following information available:1. General Information Switch modelSwitch operating system versionError numbers and messages receivedsupportSave command outputDetailed description of the problem, including the switch or fabric behavior immediatelyfollowing the problem, and specific questions Description of any troubleshooting steps already performed and the results Serial console and Telnet session logs syslog message logs2. Switch Serial NumberThe switch serial number and corresponding bar code are provided on the serial number label,as illustrated below.:*FT00X0054E9*FT00X0054E9The serial number label is located as follows: Brocade 5424 — On the bottom of the switch module.Fabric OS Troubleshooting and Diagnostics Guide53-1002751-01xv

Brocade 300, 5100, and 5300 — On the switch ID pull-out tab located on the bottom of theport side of the switch. Brocade 6505, 6510, and 6520— On the switch ID pull-out tab located inside the chassison the port side on the left. Brocade 7800 and 8000 — On the bottom of the chassis.Brocade DCX Backbone — On the bottom right on the port side of the chassis.Brocade DCX-4S Backbone — On the bottom right on the port side of the chassis.Brocade DCX 8510-4 — On the nonport side of the chassis, on the left just below the leftpower supply. Brocade DCX 8510-8 — On the bottom right on the port side of the chassis and directlyabove the cable management comb.3. World Wide Name (WWN)Use the licenseIdShow command to display the chassis’ WWN.If you cannot use the licenseIdShow command because the switch is inoperable, you can getthe WWN from the same place as the serial number, except for the Brocade DCX. For theBrocade DCX, access the numbers on the WWN cards by removing the Brocade logo plate atthe top of the nonport side of the chassis.Document feedbackQuality is our first concern at Brocade and we have made every effort to ensure the accuracy andcompleteness of this document. However, if you find an error or an omission, or you think that atopic needs further development, we want to hear from you. Forward your feedback to:documentation@brocade.comProvide the title and version number of the document and as much detail as possible about yourcomment, including the topic heading and page number and your suggestions for improvement.xviFabric OS Troubleshooting and Diagnostics Guide53-1002751-01

Chapter1IntroductionIn this chapter Troubleshooting overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Most common problem areas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Questions for common symptoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gathering information for your switch support provider . . . . . . . . . . . . . . . . . Build

Brocade 6505 switch Brocade 6510 switch Brocade 6520 switch Brocade 7800 extension switch Brocade 8000 FCoE switch Brocade VA-40FC Brocade Encryption Switch Brocade DCX Brocade DCX-4S Brocade DCX 8510-4 Brocade DCX 8510-8 What's new in this document Updated for Brocade Fabric OS v7.1.0, including .